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Preface 


The  work  presented  in  this  report  was  performed  by  L.N.K.  Corporation 
scientists  George  C.  Stockman,  Barbara  A.  Lambird,  David  Lavine,  and  Laveen 
N.  Kanal  under  contract  DAAK70-77-C-01i0  for  the  U.S.  Amy  Engineer  Topo¬ 
graphic  Laboratories,  Fort  Belvoir,  Virginia  22060.  Dr.  Robert  D.  Leighty, 
Research  Institute,  USAETL  served  as  contract  technical  monitor.  The  authors 
are  indebted  to  Dr.  Leighty  and  Mr.  George  Lukes,  Research  Institute,  USAETL 
for  helpful  interaction  and  technical  reviews  during  the  performance  of  this 
work. 


1. 


Introduction 


This  report  describes  research  performed  by  L.N.K  Corporation  for 
USAETL  during  the  approximate  period  of  1  Sept  1978  through  1  Dec  1980.  The 
study  was  directed  toward  the  use  of  knowledge  in  automatic  image  analysis 
and  toward  map-guided  image  analysis  in  particular  and  represents  a  continua¬ 
tion  of  studies  for  Wright  Patterson  Air  Force  Base  by  L.N.K.  During  the 
period  of  this  report  L.N.K.  converted  algorithms  to  run  on  a  Hewlett-Packard 
real-time  system  at  the  Research  Institute,  USAETL,  and  experimented  with  these 
algorithms  on  imagery  of  interest  to  USAETL.  The  procedures  and  results  are 
discussed  in  Sections  2  to  6. 


1.1  Problems  addressed 

Past  research  has  indicated  that  a  large  amount  of  prior  knowledge 
must  be  brought  to  bear  on  the  problem  of  interpreting  imagery  whether  it  is 
done  by  man  or  machine.  Image  analysis  by  computer  thus  leads  into  studies 
of  knowledge  representation  and  application,  which  is  currently  a  very 
active  field  in  artificial  intelligence  (A. I.)  The  work  reported  here  limits 
the  form  of  possible  knowledge  sources  to  those  which  relate  to  the  2-D  or  3-D 
spatial  geometry  of  the  real  world.  The  resulting  knowledge  is  "iconic"  and 
can  be  represented  in  a  fashion  similar  to  the  geographic  data  base  (GDB) 
that  contains  encodings  of  the  earth's  features  addressed  by  geographic  coordinates. 

By  using  iconic  knowledge,  the  image  interpretation  paradigm 


becomes  a  three  step  process.  First,  some  primitive  features  of  the 
imagery  must  be  recognized  without  any  area-specific  knowledge.  Typically 


these  features  would  be  major  lineals  or  boundaries  of  objects  such  as 
ponds  or  buildings.  Secondly,  the  imagery  is  aligned  or  registered  with 
the  GDB  by  drawing  correspondences  between  the  image  features 

and  their  iconic  analogues  in  the  GDB.  The  matching  is  formalized  by 
derivation  of  a  transformation  which  maps  points  (x,y)  of  the  image  to 
points  (u,v)  in  GDB  coordinates.  The  third  and  final  step  in  the  paradigm 
is  to  analyze  the  remaining  parts  of  the  image  which  were  not  successfully 
interpreted  in  steps  1  and  2.  This  implies  a  top-down  search  for  image 
structures  which  correspond  to  features  in  the  GDB. 

The  paradigm  is  not  the  most  general.  It  assumes  that  a  map  or 
GDB  already  exists.  Also  it  does  not  provide  for  all  forms  of  knowledge  — 
for  instance,  that  roads  tend  to  intersect  or  that  water  runs  downhill. 

However,  there  are  important  problems  to  study.  For  step  1  we  need  to 
find  feature  extraction  procedures  that  are  reliable  enough  to  detect 
major  features  in  a  variety  of  imagery.  For  step  2  we  need  to  have  a 
method  of  determining  a  global  registration  transform  using  the  ambiguous 
and  errorful  information  from  local  feature  correspondences.  Finally, 
for  step  3  a  method  of  verifying  GDB  features  in  the  imagery  is  required 
as  well  as  a  method  for  recording  positive  change  which  should  be  entered 
into  the  GDB.  These  three  problems  are  addressed  in  Sections  2,3,  and  4 
respectively.  Section  2  treats  primitive  extraction.  The  emphasis  is 
currently  on  lineal,  point  and  region  features  only.  A  method  for  automatically 
inferring  a  rotation  and  translation  transforming  image  to  map  is  given 
in  Section  3.  Classification  of  registered  regions  is  discussed  in  Section 
4.  Verification  of  lineal  GDB  features  in  grey-scale  imagery  is  introduced 


in  Section  5. 


1.2  L.N.K.  related  research  and  software  development 

More  than  2  years  of  related  research  by  L.N.K.  for  Wright 
Patterson  Air  Force  Base  has  been  reported  in  Stockman  [1979].  The  techniques 
and  software  developed  in  that  study  were  carried  forward  into  the  current 
research.  Thus  results  reported  here  represent  a  second  iteration  of  improve¬ 
ment  of  techniques  and  testing  on  a  wider  variety  of  images.  Prior  work  was 
done  on  digitized  images  stored  on  disk  files.  This  report  describes  image 
processing  performed  using  on-line  analogue  image  storage  sampled  by  a  32x32 
pixel  digital  array  sensor  positioned  under  program  control.  The  three  soft¬ 
ware  subsystems  implemented  at  USAETL  are  the  feature  detection  subsystem  dis¬ 
cussed  in  Section  2,  the  registration  subsystem  discussed  in  Section  3,  and 
the  verification  subsystem  documented  in  Section  5.  These  subsystems  are 
detailed  in  the  Appendices. 


1.3  Sample  data  sets 

The  experiments  discussed  in  Sections  2  thru  5  used  the  images 
shown  in  Figures  1.1  thru  1.3.  Tables  1.1  and  1.2  give  coordinates 
of  selected  points  labeled  on  the  images.  This  data  was  chosen  because  of 
rich  straight  line  structure  which  the  existing  techniques  had  been 
developed  to  handle.  Terrain  with  less  man-made  structure  will  be 
handled  later  when  feature  extraction  techniques  are  further  developed. 
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Table  1.1  Digitization  of  points  on  4621  image  using  Talos  digitizer, 

1(0.001  inch  resolution)  and  scanning  stage  (0.0005  inch 

resolution) 
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X  Y 

DESCRIPTION 

1 

2498 

3641 

5000 

5000 

X  (origins) 

2 

4365 

2747 

8722 

6803 

T 

3 

5583 

219] 

11120 

8052 

X 

4 

3044 

4744 

3913 

7203 

T 

5 

3342 

5357 

3314 

8405 

T 

6 

3611 

5908 

2814 

9511 

T 

7 

1980 

6678 

- 

- 

T 

8 

3975 

5750 

3508 

9946 

T 

9 

5516 

5073 

6563 

11448 

T 

10 

7130 

3514 

10964 

12108 

T 

11 

7430 

4091 

10367 

13304 

L 

12 

7723 

4700 

9866 

14600 

L 

13 

8642 

4258 

- 

- 

X 

14 

6952 

3223 

11209 

11461 

A 

15 

6942 

3613 

10565 

12000 

L 

16 

7233 

4181 

10010 

13101 

T 

17 

4861 

7575 

1710 

13600 

X 

18 

5980 

7667 

- 

- 

Y 

19 

6444 

7480 

- 

- 

T 

20 

9471 

6190 

- 

- 

T 

21 

5013 

7886 

1399 

14295 

A 

22 

4480 

7729 

- 

- 

T 

23 

7346 

9632 

- 

- 

A 

24 

9282 

5763 

- 

- 

A 

25 

9700 

6104 

- 

- 

Y 

26 

5279 

7999 

1513 

14804 

A 

27 

4010 

1991 

9388 

5480 

A 

28 

5480 

2552 

- 

- 

L 

29 

4808 

2527 

- 

- 

A 

Figure  1.2 


Aerial  photo  over  Franc; 
"DRF.UX  13".  (See  Figure 
stereo  mate. ) 


referred 
1  .  5 
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Figure  1.3  Aerial  photo  over  France  referred  to  as 
"DRFIJX  12"  (See  Figure  1.2  for 

stereo  mate) . 


Table  1.2  Digitization  of  points 
0.001  inch  resolution.) 

IMAGE  PT  it  DREUX  13  DREUX  12 
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X 
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8141 
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7004 
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on  Dreux  image  (using  Talos  digit! 


DESCRIPTION 


field  corher  (origin) 

Y  road  intersect 

Y  road  intersect 
T  road  intersect 

X  intersect ,  road  &  RR 

nose  point  on  road 
comer  of  dark  field 
comer  of  dark  field 
comer  of  dark  field 
T  road  intersect 

X  or  T  road  intersect 
T  road  intersect 

Y  road  intersect 
T  road  intersect 
T  road  intersect 

X  intersect ,  road  &  RR 
T  road  intersect 
X  road  intersect 

Y  road  intersect 

Y  road  intersect 

T  road  intersect 

Y  road  intersect 
T  road  intersect 

on  straight  of  road 
on  straight  of  road 

on  straight  of  road 


1.4 


Laboratory  configuration,  parameters,  and  definitions 

The  laboratory  environment  used  for  the  experiments  is  described 


here  and  is  diagrammed  in  Figure  1.6.  As  Figure  1.6  shows,  the  ROSA  system 
allows  a  computer  program  to  directly  interact  with  an  image  stored  on  film. 

A  small  window  on  the  film  (about  1/8")  can  be  imaged  onto  a  32  by  32  element 
photo  diode  array  which  is  then  sampled  and  converted  to  10-bit  digital  values 
for  consumption  by  the  program.  The  end  result  is  1024  integer  values  in  a 
2-D  FORTRAN  array.  Using  executive  calls  a  FORTRAN  program  can  position  the 
stage  to  any  point  in  the  selected  5  inch  x  5  inch  area  and  on  a  CRT  display 
the  user  can  view  the  current  window. 

Using  a  mirror  as  a  switch,  the  laser  beam  passing  through  the 
film  could  be  sent  to  an  electro-optical  subsystem  which  computes  the  Fourier 
Transform  and  makes  it  available  to  the  program.  The  term  "ROSA"  originally 
meant  "Recording  Optical  Spectrum  Analyzer"  and  is  till  used  although  not 
descriptive  of  the  current  optical/digital  sampling  system  which  has  evolved. 

Table  1.3  contains  a  summary  of  the  parameters  and  definitions  used 
in  the  image  processing  environment.  Note  the  difference  in  resolution  of  the 
different  units  of  hardware.  The  stage  used  to  move  the  film  is  driven  in 
units  of  one-half  mil.  The  photo-diodes  of  the  array  are,  however,  on  4  mil 
centers.  The  x-y-digitizer ,  on  the  other  hand,  has  a  resolution  of  1  mil. 

Thus  there  are  many  scale  changes  required  by  a  program  which  measures  objects 
on  the  film  and  relates  them  to  objects  digitized  by  hand  from  a  paper  map. 


[ 


Figure  1.6  Diagram  of  ETL  ROSA  real  time  image 
sampling  system. 


Table  1.3 


Parameters  and  definitions  for  ETL  ROSA  Lab  environment 

mil  one  thousandth  of  an  inch 

pixel  a  10-bit  integer  measuring  the  intensity  transmitted  by 

a  4  mil  x  4  mil  square  portion  of  film. 

array  a  32  x  32  2-D  array  of  light  sensitive  detectors  used 

to  sample  film  and  create  a  32  x  32  element  2-D  matrix  of 
pixels  for  computer  processing. 

stagel  stage  element  or  resolution  of  stage  movement  which  is 

0.5  mil.  Each  pixel  is  thus  8  stagels  on  a  side. 

window  that  portion  of  the  film  that  can  be  sampled  at 

a  given  stage  position;  representing  an  area  0.128  x  0.128 
inches  square  or  32  x  32  pixels  each  4  mils  on  a  side. 

registration  obtaining  a  transformation  T(x,y)  =  (u,v)  which  transforms 

an  image  point  (x,y)  into  its  corresponding  map  point  (u,v). 
In  this  report  T  consists  of  a  rotation  0  with  O°<0<36O° 
and  a  translation  no  more  than  half  the  image  diameter. 
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1.5 


Images  placed  on  the  stage  were  always  addressed  in  stage  coordi¬ 
nates  with  the  "center"  pixel,  x=16  and  y=16,  denoting  the  window  position. 
An  outstanding  pass  point  was  selected  as  a  logical  origin  for  each  pic¬ 
ture  and  computer  processing  was  always  begun  with  this  point  in  the  cen¬ 
ter  of  the  window.  The  results  reported  here  were  often  obtained  by  making 
several  different  runs  with  possible  removal  of  the  image  from  the  stage 
in  the  interim.  For  the  image  4621  the  point  #1  was  chosen  as  the  logical 
origin  and  was  assigned  coordinates  x=5000,  y=5000,  as  shown  in  Table  1.1. 
Clearly,  repeatability  of  the  image  addressing  is  dependent  on  the  care 
taken  to  mechanically  set  up  the  film  and  viewing  equipment  (stage,  mount, 
mirror,  etc)  and  is  probably  no  better  than  about  0.01  inch  for  the  variety 
of  experiments  reported. 
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2. 


Feature  Detection 


Most  L.N.K.  studies  have  been  concerned  with  lineal  and  point 
features  only;  regional  analysis  is  addressed  somewhat  later  in  the  sec¬ 
tion.  By  lineal  features  we  mean  continuous  curves  which  define  the 
boundary  between  regions  of  tonal  contrast.  Usually  these  curves  define 
a  feature  of  interest  to  the  mapping  community  such  as  a  stream,  road,  or 
side  of  a  building.  Field  boundaries  can  also  yield  prominent  lineals 
in  an  image  and  may  or  may  not  be  of  interest  in  the  interpretation  of 
imagery . 


The  beginning  parts  of  this  section  of  the  report  are  devoted  to 
edge  detection.  Here,  the  concept  of  lineal  is  almost  synonomous  with  the 
term  "edge"  used  in  the  literature.  The  fine  difference  is  that  a  true 
lineal  feature  (i.e.  a  road  or  river)  is  2-D  and  actually  is  characterized 
by  2  edges,  one  on  each  side  of  the  road  for  instance. 

Point  features  are  defined  from  certain  lineal  features.  For 
example,  points  of  intersection  or  of  high  curvature  can  provide  unique 
features  of  the  image  for  registration  and/or  interpretation.  Point  fea¬ 
tures  can  also  be  defined  as  the  centers  of  small  identifiable  regions 
such  as  buildings  or  ponds. 

By  using  only  lineals  and  points  a  very  efficient  image  abstrac¬ 
tion  is  rendered  which  can  be  of  immediate  use  in  registering 
images  or  in  the  partial  interpretation  of  imagery.  Some  very  successful 
registration  experiments  using  only  edge  or  point  data  are  reported  in 
Section  3.  The  rest  of  this  section  treats  edge,  point,  and  region  extrac¬ 
tion. 
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2.1 


Gradient  Operator  and  Masking 

A  point  can  be  discovered  to  be  an  edge  point  by  testing  the 
tonal  values  in  halfplanes  on  either  side  of  the  point.  Figure  2.1  shows 
the  32  directions  for  edges  that  were  used  in  the  reported  research.  A 
general  purpose  routine  (RPSML)  exists  which  can  diff erentitate  either 
8,  16  or  32  directions  around  the  circle.  Three  "masks"  for  computing 
directional  gradient  values  are  shown  in  Figure  2.2.  Direction  d  =  1  is 
the  vertical  direction  with  higher  tones  at  the  right  while  direction  d  =  7 
is  nearly  horizontal  with  higher  tones  below.  Given  a  pixel  (x,v)  in  the 
image,  the  gradient  magnitude  in  each  direction  d  =  1,2,..., 32  can  be 
computed  by  adding  and  subtracting  tonal  values  as  indicated  by  the  masks 
in  Figure  2.2.  Division  by  a  normalization  factor  is  performed  to  take  into 
consideration  the  number  of  pixels  used  and  to  make  the  magnitude  geometri¬ 
cally  isotropic.  The  magnitude  and  direction  of  the  gradient  at  (x,v)  is 
taken  to  be  the  magnitude  and  direction  where  a  maximum  is  achieved.  Note 
that  for  d  =  17  the  mask  for  d  =  1  could  be  used  with  reversal  of  the  sien 
on  the  magnitude,  so  that  only  16  masks  are  actually  applied.  Only  4 
directions  are  tried  at  resolution  8.  The  points  selected  for  the  masks 
sometimes  differ  from  the  ideal  due  to  implementation  considerations. 

Points  of  weak  gradient  magnitude  may  be  detected  from  a  repre¬ 
sentation  of  the  image  as  shown  in  Figure  2.3  which  shows  the  results  of  a 
gradient  computation  on  an  image  of  an  airplane  wing.  The  arrows  indicate 
gradient  direction  and  show  that  the  background  tones  are  of  higher  value 
than  the  tones  on  the  wing  itself.  The  lower  edge  of  the  wing  has  orienta¬ 
tion  of  about  13  while  those  on  the  top  edge  are  roughly  30.  Note  that  the 
"USAF"  symbols  on  the  wing  create  much  structured  gradient  activity  but  that 


resolution  is  not  fine  enough  for  recognition. 
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Figure  2.1 
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Figure  2.  2  Three  masks  used  for  computing  the  gradient  at 

a  point  in  directions  1,4,  and  7.  The  directional 
resolution  is  one  32nd  of  the  circle;  N  is  the 
normalization  factor.  Edge  directions  of  1,4, 
and  7  are  indicated  by  the  arrows. 
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Figure  2.3  Gradient  direction  of  high  contrast 
points  of  right  airplane  wing. 
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Quite  often,  the  presence  of  one  edge  will  interfere  with  the  detection 
of  a  neighboring  edge.  This  is  particularly  true  if  only  a  limited 
fraction  of  the  image  points  with  high  gradient  magnitudes  are  selected 
to  represent  the  image.  In  order  to  search  for  edges  in  the  neighborhood 
of  previously  detected  edges  we  could  suppress  the  known  edge  points  to 
remove  the  interference  with  the  detection  of  other  edge  points.  Conversely, 
to  extend  an  existing  edge  segment  we  could  suppress  points  of  incompatible 
gradient  directions  in  order  to  enhance  the  detection  of  continuing  edge 
points . 

Point  masking  can  be  done  when  the  gradient  image  is  computed  by 
applying  either  locational  or  directional  constraints  (masks)  as  shown 
in  Figure  2.4.  A  program  EDGEY  (see  Appendix  C)  uses  such  masking  - 
first  to  extend  existing  edges  by  suppression  of  competing  gradient 
directions  and  then  to  suppress  the  existing  edges  in  order  to  detect 
intersecting  edges.  In  fact,  to  detect  road  intersections,  two  direc¬ 
tional  masks  must  be  set,  one  for  each  side  of  the  known  road. 


Point  (x,y)  where 
alx  +  bjy  +  c1  < 


0 


0 


Point  (x,y)  with  gradient 
d 

direction  d  such  that 
d^  d  ~  can  be 

masked  off. 

-*d2 


(a)  Points  near  a  known  edge  (AX, AY)  -  (BX,BY) 
can  be  masked  off  by  their  location  with 
respect  to  two  halfplanes  or 

(b)  points  with  gradient  direction  in  a 
certain  range  can  be  masked  off. 
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2.2 


Continuous  Edge  Linking 

By  examining  the  neighbors  of  edge  pixels  it  is  sometimes  possible 


I 


to  determine  the  two  pixels  which  continue  the  edge  in  the  forward  and 
backward  direction.  (Forward  edge  traversal  by  definition  keeps  higher 
tonal  values  toward  the  right.)  Continuation  can  be  determined  from  the 
spatial  orientation  of  the  neighbors  and  the  gradient  direction  of  the 
neighbors,  all  relative  to  that  of  the  original  pixel. 

When  determining  which  neighbor  is  the  best  forward  and  backward 
continuation  a  variable  sized  neighborhood  is  scanned  in  a  spiral  pattern 
for  the  first  satisfactory  neighbor.  See  Appendix  B  (/RPSCH/  common)  for 
definition  of  the  spiral  pattern. 

Stockman  [1979J  contains  a  detailed  discussion  of  continuous  edge 
linking.  Figure  2.5  shows  best  forward  and  backward  linking  done  for 
all  pixels  shown  in  Figure  2.3.  (Done  by  program  RPSWK. )  A  second  global 
process  can  extract  long  chains  of  points  which  mutually  link  to  each 
other  and  which  have  above  threshold  length.  Figure  2.6  shows  the  long 
chains  derived  from  the  data  in  Figure  2.5.  (Program  RPSLK  extracts  the 
chains. ) 

Chains  of  15  or  more  pixels  were  extracted  from  the  imagery 
sampled  in  the  ETL  ROSA  lab.  Often  these  chains  formed  a  straight 
edge  which  was  usually  also  detected  by  the  Hough  detector  (Section  2.3). 
Since  interesting  curved  edges  were  not  being  formed  due  to  the  small  32x32 
window  size,  the  results  of  curve  extraction  were  not  used  for  further 
processing.  In  future  experiments  the  effective  window  size  could  be  en¬ 
larged  and  more  interesting  curved  edges  should  be  obtained. 


Plot  of  all  forward  and  backward  linking 
relationships  among  high  contrast  points 
of  Figure  2.3. 


Figure  2.6  Long  curve  segments  extracted  from 
related  points  of  Figure  2.5. 
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2 . 3  Hough  detection  of  straight  edge  elements 

The  Hough  transform  Is  a  popular  device  for  the  detection  of 
linearity  in  a  set  of  points.  The  mathematical  development  and  practical 
use  of  the  Hough  transform  is  discussed  in  Duda  [1972]  and  Stockman  [1977]. 
The  Hough  detector  was  used  heavily  to  achieve  the  results  reported 
here  and  proved  to  be  very  reliable.  The  strategy  used  was  to  set 
stringent  thresholds  so  that  the  false  alarm  rate  would  be  nearly  zero. 

In  fact,  no  false  alarms  have  been  documented  although  not  all  of  the 
thousands  of  windows  processeJ  have  been  studied.  There  were  some 
"unwanted"  detections  on  faint  image  structure  such  as  field  mowings  and 
buildings  which  were  difficult  to  discern  on  the  CRT. 

The  windows  sampled  were  32x32  pixels,  of  which  only  the  center 
24x24  pixels  were  useable  to  the  resolution  32  gradient  operator,  so 
detection  thresholds  were  set  at  either  15  or  20  meaning  that  15  or  20 
pixels  had  to  line  up  before  a  detection  was  signaled.  Figure  2.7 
documents  the  detection  of  a  linear  edge  segment  between  a  road  and  a 
field.  Gradient  directions  21  to  25  have  been  masked  out  so  that  one 
side  of  the  road  is  nearly  suppressed  —  the  side  barely  survives 
due  to  jitter  in  the  gradient  directions .The  gradient  magnitude 
histogram  is  shown  which  was  used  to  select  38  points  to  represent  the 
image  window.  The  38  points  are  all  visible  in  the  grey  shade  plot  of 
the  gradient  image.  The  coarse  accumulator  array  is  defined  for  the  32 
angular  directions  and  radius  values  from  -12  to  +12:  thus  there  are 
32  x  25  =  800  accumulators.  Each  of  the  selected  38  points  can  indicate 
an  incrementing  of  those  few  accumulators  defined  by  an  R  and  P  compatible 
with  the  gradient  direction  of  the  point.'  The  entire  set  of  resulting 
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Figure  2.7 


Point  selection  and  Hough  detection 
of  straight  edge  along  road. 


accumulators  is  pictured  in  Figure  2.8.  One  side  of  the  road  is  detected 
by  peak  detection  in  the  accumulators  at  R  =  -3  and  0  =  292°.  0  =  292° 
because  that  Is  the  true  geometric  direction  of  tonal  increase  across  the 
edge  and  R  =  -3  because  the  edge  is  actually  located  3  units  in  the 
direction  292  -  180  from  the  origin  at  pixel  (16,16).  The  Hough  detection 
is  focused  by  contributing  the  38  points  to  a  refined  accumulator  array 
with  only  5x7  accumulators.  In  this  process  the  edge  location  is  refined 
to  R  =  -3  and  0  =  292°.  The  peak  response  is  diminished  from  18  to  14 
because  the  width  of  the  edge  "template"  is  now  only  1  instead  of  3  pixels. 
The  detected  edge  element  is  reported  to  lie  between  points  (1,13)  and 
(32,28)  in  the  image  window  and  (6773,5897)  and  (7021,6017)  in  terms  of 
global  stage  coordinates. 

The  focused  accumulator  uses  2°  angular  resolution  but  this  resolu¬ 
tion  is  not  really  obtainable  with  32x32  windows.  The  2°  unit  was  designed 
for  the  predecessor  system  which  used  64x64  windows  where  lines  2°  apart 
differed  by  several  pixels  along  their  extent.  In  the  research  reported 
here,  Hough  detections  were  made  in  batch  mode  by  sampling  the  imagery 
with  a  raster  scan  of  butting  windows.  (Program  EDGEX  is  detailed  in 
Apperadix  C. ) 
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Figure  2.8  Coarse  arui  fine  accumulator  arrays 
showing  response  to  road  edge  shown 
in  Figure  2.7. 
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2.3.1.  Experiments  with  Primitive  Detection  of  Edge  Elements 

The  Hough  detector  was  applied  to  over  a  thousand  windows  on  the  Images 
4621  and  DREUX  13  (see  Figures  1.1  and  1.2).  Straight  edge  elements  detected 
from  4621  are  plotted  in  Figure  2.9  (a)  while  detections  from  DREUX  13  are 
plotted  in  Figure  2.9  (b) .  Parameters  used  for  the  detectors  are  given  in 
Table  2.1.  It  should  be  noted  that  stringent  thresholds  were  applied, 
especially  on  the  number  of  points  on  a  straight  line  so  that  few  false 
alarms,  if  any,  were  reported.  Only  the  straight  edge  elements  are  plotted; 
the  curves  extracted  by  the  edge  linking  routine  were  ignored. 

Many  detections  are  evident  along  the  major  roads  in  4621.  The 
vertically  appearing  highway  along  the  right  edge  of  the  image  is  well 
covered  while  the  other  roads  in  the  image  have  only  spotty  coverage. 

Missed  detections  along  the  roads  are  attributed  to  one  or  more  of  the 
following  effects: 

(1)  unfavorable  contrast  due  to  sun  angle  and  shadows, 

(2)  gradient  direction  falls  in  between  two  of  the  32  coarse  level 
directions  used  by  the  Hough  detector, 

(3)  road  cuts  across  a  corner  of  the  sampling  window  and  not 
enough  points  exist  to  trigger  detection, 

(4)  road  cuts  through  center  of  window  and  the  5%  of  the  strongest 
gradient  points  are  equally  distributed  on  both  sides  of  the 
road  such  that  neither  edge  passes  threshold. 

(5)  in  the  neighborhood  of  other  edges,  such  as  other  roads  or 
field  boundaries,  the  effect  of  (4)  is  compounded  further. 
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2.9(a)  Plot  of  straight  image  n'ges  extracted 
from  4621. 
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Table  2.1 


Details  of  primitive  edge  extraction  experiments 
using  4621  and  Dreux  13  (Program  EDGEX.) 


window  size 
window  spacing 
it  points  selected 
min  gradient  threshold 
//points  on  line  (coarse) 
it  points  on  line  (fine) 


32x32  pixels 
250  stagels 
5% 

8 

20 

14 

15 


it  points  on  curve 


Several  isolated  detections  that  appear  to  be  noise  were  made  on  drainage 
ditches  or  mowing  lines  in  fields.  199  detections  were  made  overall; 
roughly  half  by  the  Hough  detector. 

While  the  primitive  edges  from  4621  produce  a  poor  rendition  of 
4621  image  structure,  they  nevertheless  permit  human  recognition  of 
the  region  portrayed.  This  is  not  true  of  edge  elements  extracted 
from  the  Dreux  image  and  plotted  in  Figure  2.9(b).  The  boundaries  between 
a  few  fields  are  nicely  covered  but  most  of  the  detections  are  isolated 
from  others  so  that  the  human  eye  does  not  see  global  linearity;  for 
instance  the  roads  are  not  apparent  due  to  their  change  of  direction  and 
spotty  coverage.  262  detections  were  made  overall,  about  half  by  the 
Hough  detector  and  half  by  the  curve  linking  procedure.  However,  despite 
the  lack  of  structure  evident  to  a  human,  there  was  ample  structure  for 
the  automatic  registration  procedure  to  align  the  image  with  a  map 
of  the  area.  These  registration  results  are  presented  in  Section  3. 

Also,  the  primitive  edge  elements  were  used  as  input  by  another  procedure 
which  extended  the  edge  segments  and  searched  for  intersections  along 
them  as  described  later. 


An  experiment  was  performed  to  see  if  the  ROSA  frequency  domain  detector 
could  find  edges  as  well  as  the  Hough  edge  detector.  The  Hough  detector  had 
previously  been  applied  to  the  image  4621  and  the  positions  of  the  extracted 


edges  were  recorded.  Fifty  of  these  detections  were  randomly  selected  for 
sampling  by  the  ROSA  detector.  Thirty  control  positions,  where  the  Hough 
detector  found  no  edges,  were  also  selected  for  sampling.  The  32  wedge  sign¬ 
atures  for  each  sample  were  examined  in  order  to  find  a  scheme  for  determin¬ 
ing  whether  an  edge  existed  in  the  sample. 

The  first,  and  as  it  turns  out,  the  best  classification  scheme  was  to 
use  the  ratio  of  the  maximum  wedge  value  to  the  minimum  wedge  value.  If  this 
ratio  was  less  than  2.5,  then  the  sample  was  said  to  contain  no  edges;  if  the 
ratio  was  greater  than  2.5,  then  the  sample  was  said  to  contain  an  edge. 

Using  the  Hough  detector  as  a  standard,  this  technique  erroneously  classified 
one  non-edge  sample  as  an  edge  sample.  The  histogram  of  the  ratios  of  the 
two  groups  is  shown  in  Figure  2.10(a). 

More  elaborate  classification  techniques  were  also  tried,  but  none  of  these 
worked  as  well.  For  example.  Figure  2.10  also  shows  the  histograms  of  the 
ratios  of  the  maximum  wedge  value  divided  by  the  median  wedge  value,  the 
upper  octile  wedge  value  divided  by  the  median  wedge  value,  and  the  upper 
quartile  wedge  value  divided  by  the  median  wedge  value.  None  of  these  methods 
separated  the  two  groups  as  well  as  the  first  method  did. 

Other  ratios  such  as  dividing  the  upper  quartile,  upper  octile,  and  the 
median  by  the  maximum  were  tried.  In  addition,  comparing  the  percentage  dif¬ 
ference  between  these  three  ratios  were  also  examined.  These  methods  separated 
the  groups  even  more  poorly. 

Since  computing  the  ratio  between  the  maximum  and  minimum  wedge  value  is 


l 


E 

I 

!  very  simple,  and  the  position  of  the  maximum  wedge  determines  the  orientation 

of  the  edge,  it  is  felt  that  the  ROSA  detector  could  efficiently  be  used  as  an 

edge  detector. 


1 

I 
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(a)  Histogram  of  ratio  of  maximum  wedge  value  to  minimum  wedge  value. 
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(b)  Histogram  of  ratio  of  maximum  wedge  value  to  median  wedge  value. 
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Figure  2.10.  Histograms  of  various  classification  schemes  to  differentiate 
between  a  non-edge  group  (lower  shown)  and  edge  group  (upper 
group  shown) . 


2.5  Edge  extension 

Edges  sometimes  fade  away  and  sometimes  continue  to  meet  other 

edges.  For  instance,  a  road  may  end  in  a  field  but  is  perhaps  more 

likely  to  end  by  meeting  another  road.  Once  a  straight  edge  segment  is 

detected,  it  is  therefore  wise  to  attempt  to  track  its  full  extension. 

A  program  was  written  to  attempt  to  iteratively  extend  a  detected 

straight  edge  in  both  directions.  (The  program  is  EDGEY  and  is  detailed 
in  Appendix  C.)  In  attempting  the  extension,  all  competing  gradient 

directions  are  masked  off  so  that  the  edge  is  not  lost  in  a  background 

of  competing  edges.  The  image  window  is  positioned  so  that  the 

predicted  extension  segment  would  pass  through  its  center:  thus  any 

detection  should  have  R  =  0. 

Primitive  detections  in  the  32x32  window  represent  only  1/8 

i 

* 

inch  on  the  actual  film.  In  some  cases  documented  in  section  2.7  extensions 
of  up  to  an  inch  or  more  were  made.  It  is  possible  to  extend  this  technique 
so  that  it  could  be  applied  to  curved  edges.  While  straight  edges  are 
extended  or  are  found  to  terminate,  it  is  appropriate  to  check  for  inter¬ 
secting  edges  as  described  in  the  next  section. 
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2. Intersection  detection 

Edge  intersection  points  are  highly  desirable  as  photo  pass  points 
and  are  thus  a  target  for  automatic  recognition  effort.  Moreover,  we 
would  like  to  describe  the  intersection  as  much  as  possible  so  that  it 
can  be  differentiated  from  other  intersections.  Figure  2.11  shows 
a  catalog  of  possible  intersection  types  which  we  might  like  to  recognize 
automatically.  It  was  decided  that  first  an  algorithm  should  be 
developed  to  detect  arbitrary  intersections,  then  further  testing  could 
be  used  to  classify  the  detected  intersection.  Toward  this  end  a 
simple  routine  was  implemented  to  test  for  edge  activity  nearly  per¬ 
pendicular  to  an  existing  edge.  In  fact,  intersections  would  be  checked 
for  while  extending  existing  edges.  (Program  '  EDGEY  again).  Each 
existing  edge  element  is  placed  into  one  of  4  classes  as  shown  in 
Figure  2.12.  Then  the  window  can  be  positioned  once  on  each  side 
of  the  edge  so  that  the  Hough  detector  can  detect  intersecting  edges. 

Of  course  the  current  edge  direction  as  well  as  its  180°  supplement^ 

\ 

are  masked  off  during  detection.  The  two  window  placements  are  made 
every  100  or  so  stagels  along  the  existing  edge  so  that  no  neighboring 
region  is  ignored. 


36 


Figure  2.11  Catalogue  of  possible  local  elementary 
features  for  recognition  &  registration. 


2.6.1  Experiments  with  edge  extension  and  intersection  detection. 

In  the  first  experiment  with  intersection  detection,  the 
gradient  masking  technique  was  tested  to  see  if  masking  of  a  major 
edge  would  allow  detection  of  weaker  intersecting  edges.  Certain  lineal 
features  o';  4621  are  shown  in  Figure  2.13.  Several  search  tracks  along 
certain  o';  these  lineals  were  selected  for  intersection  detections. 

Searches  were  conducted  between  the  following  pairs  of  points: 

—  point  1  to  point  6,  point  6  to  point  11,  point  8  to  point  21, 
point  1  to  point  29,  point  3  to  point  14,  and  point  15  to  point  16. 

Windows  were  sampled  at  intervals  of  150  stagels  along  each  lineal 
segment  and  on  both  sides  of  the  lineal  as  shown  in  Figure  2.12, 

For  all  windows  the  two  gradient  directions  normal  to  the  track  were 
masked  off  to  remove  contention  from  the  major  edge  points.  If  a 
detection  was  made  on  one  side  only,  then  a  'V  intersection  was  reported. 
If  a  detection  was  made  on  both  sides,  an  "X"  intersection  was  reported. 

Results  of  intersection  detection  are  given  in  Table  2.2 
Some  of  the  desired  connecting  roads  and  driveways  were  detected 
(i.e.  at  points  1,3,4,5,6,8,11  and  17)  and  some  were  missed  (i.e.  at 
points  2  and  9) .  Many  field  boundaries  wt~e  picked  up  as  well  as  some 
driveways  not  numbered  in  Figure  2.13. 

The  program  was  modified  to  extend  edges  and  was  run  on  the 
DREUX  13  data.  The  results  are  plotted  in  Figure  2.14.  Detected 
intersections  are  indicated  by  the  small  circles.  Extended  edges  are 
also  evident  in  the  plot.  Some  edge  elements  were  lengthened  from  1/8 
inch  to  an  inch  or  so. 
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Table  2.2 


Results  of  Connecting  Edge  Experiment  on  linage  4b 


Intersection  Type  of  Description 

in  Stage  Coord.  Intersect. 


tracking  from  point  #1  to  point  #  6  steps  of  150  stagels 


(4999,5011)  T 
(4937,5137)  T 
(4850,5317)  T 
(4828,5362)  T 
(4691,5642)  T 
(4499,6032)  X 
(3997,7060)  T 
(3913,7242)  T 
(8762,9010)  T 
(3612,7849)  T 
(3325,8436)  T 
(3029,9041)  T 
(2893,9320)  T 
(2788,9533)  T 


point  ill  (really  an  X) 

road/field 

road/f ield 

road/ field 

road/field 

road/drainage 

road/ field 

point  //  4  road/driveway 

field  boundary/road 

field/road 

point  it  5 

road/field 

?  investigate 

point  it  6 


tracking  from  point  It  6 
(3100,9654) 

(2888,9548) 

(3569,9888) 

(4046,10127) 

(4698,10453) 

(6020,11113) 

(6942,11575) 

(7063,11635) 

(6986,11596) 

(8276,12241) 

(8705,12456) 

(10480,13344) 
tracking  from  point  it  8  to  point  it  21 
(3526,9900) 

(3240,10502) 

(3103,10786) 

(1756,13584) 

(1557,13997) 

track  from  point  It  1  to  point  It  29 
(5008,5003)  X 

(6765,5849)  T 

(6706,5821)  T 

(9100,6980)  T 


to  point 

T 

T 

T 

T 

T 

T 

T 

T 

T 

T 

T 

T 


T 

T 

T 

X 

T 


it  11 

road/ field 

point  it  6 

point  it  8 

woods? 

road/field 

road/driveway 

driveway  to  buildings 

buildings 

buildings 

road/driveway  (unnumbered  point) 
road/driveway  (unnumbered  point) 
point  it  11 


point  it  8  again 
road/field 
road/field 
point  it  17 
road/ field 


point  it  1 

noise,  threshold  set  too  low 
noise,  threshold  set  too  low 
road/field 


t 
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Table  2.2  (continued) 


Intersection 
in  Stage  Coord 


track  from  point 

(11120,8072) 

(11147, 9416( 

(11153,9716) 

(11158,9975) 

(11170,10546) 

(11179,10989) 

(11190,11531) 

(11191,11589) 

track  from  point 

(10454,12220) 

(10415,12296) 

(10112,12892) 


Type  of  Description 

Intersect . 


3  to  point 

It  14 

T 

point  It  3  (really  an  X) 

T 

road/treeline 

T 

road/treelihe 

T 

road/f ield 

T 

road/treeline 

T 

road/ field 

T 

■? 

T 

actually  same  road  bending 

15  to  point 

It  16 

T 

small  driveway 

T 

road/ trees 

T 

.  road/trees 
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2 . 7  Intersection  Classification 

Apart  from  the  work  described  in  Section  2.6,  an  intensive  investiga¬ 
tion  into  the  detection  and  classification  of  intersections  given  an  initial 
set  of  detected  line  segments  was  conducted.  This  work  is  easily  integrated 
into  the  feature  detectors  EDGEX  and  EDGEY. 

The  detection  and  classification  of  intersections  given  a  set  of  detected 
line  segments  was  divided  into  three  steps.  The  first  step  was  to  merge  line 
segments  with  approximately  the  same  orientation  and  position.  The  second 
step  was  to  pair  line  segments  which  were  180°  off  in  orientation,  but  very 
close  to  each  other.  This  step  would  associate  the  two  sides  of  a  road  as 
a  single  entity.  The  last  step  was  to  take  the  resulting  paired  line  seg¬ 
ments  and  leftover  unpaired  line  segments  and  determine  whether  any  of  them 
intersected  and  classify  the  intersections  as  shown  in  Figure  2.11.  The 
software  for  the  classification  is  presented  in  Appendix  F. 

2.7.1  Merging  Line  Segments 

The  goal  of  the  first  step  is  take  a  set  of  detected  line  segments  and 
to  merge  line  segments  which  have  the  same  approximate  position  and  orienta¬ 
tion.  The  three  possible  types  of  situations  where  merging  should  be  done 
are  shown  in  Figure  2.15.  The  first  case  is  called  linking,  i.e.  one  line 
segment  can  be  extended  and  merged  with  the  second.  The  second  and  third 
cases  are  called  domination,  i.e.  where  one  image  edge  was  detected  twice  in 
slightly  different  locations.  This  can  happen  when  overlapping  edge  detec¬ 
tion  windows  are  used. 

The  merging  algorithm  steps  are: 

(1)  for  each  line  segment  v  compare  with  each  of  the  other  line  seg- 
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Figure  2.15.  The  three  cases  where  merging  of  line  segments  should 
be  done. 


Figure  2.16.  In  order  for  merging  to  occur,  the  indicated  distances 
must  be  less  than  a  supplied  tolerance. 


merit  v  in  order  of  increasing  distance  between  and 


v,.  This 


ordering  is  determined  by  calculating  tbe  (R,0)  form  of  the  line 
segment  [see  Duda  and  Hart,  (1972)  Hough  edge  detector]  and  com¬ 
puting  the  differences  between  the  polar  radii,  R,  of  each  pair. 
The  comparison  is  done  with  closest  line  segments  first,  since 
the  result  of  combining  three  or  more  edge  segments  was  dependent 
upon  the  order  of  combination. 

(2)  For  each  pair  (v.,v  ),  it  was  determined  if  linking  or  domination 

1  j 

would  be  needed.  For  linking  to  be  indicated  (a)  the  angle 
between  and  Vj  had  to  be  less  than  AO  tolerance,  (b)  the 
distance  between  the  endpoints  and  the  other  line  had  to  be  less 
than  As^,  and  (c)  the  linking  separation  had  to  be  less  than  * s 2 , 
as  shown  in  Figure  2.16  (a).  If  domination  was  indicated,  then, 
in  order  for  the  pair  to  be  merged,  the  distance  between  the  end¬ 
points  and  the  other  line  had  to  be  less  than  As  ,  as  shown  in 
Figure  2.16  (b). 

(3)  Finally  if  the  line  segment  pair  satisfied  the  merging  test,  they 
were  merged  and  replaced  with  the  new  merged  line  segment. 

Several  methods  of  determining  the  endpoints  of  the  new  merged  line  seg¬ 
ment,  for  the  cases  shown  in  Figure  2.15  (a)  and  (b),  were  tried.  The  final 
method  found  the  slope  of  the  new  line  by  averaging  the  slope  of  the  two  old 
segments.  A  point  on  the  new  line  was  found  by  averaging  the  endpoints  of  the 
two  old  line  segments.  The  endpoints  of  the  new  line  segment  were  calculated 
using  the  extremes  of  the  old  line  segments.  Note  that,  in  averaging  two 
almost  vertical  lines  whose  slopes  are  opposite  in  sign,  the  average  slope 
should  tend  towards  infinity  instead  of  zero.  This  was  accomplished  bv 


averaging  the  inverse  of  the  slopes  and  then  taking  the  inverse  of  the  re- 


suit.  The  extremes  used  to  find  the  new  endpoints  were  either  the  extreme 


x-coordinate  values,  if  the  slope  was  less  than  1.0,  or  the  extreme  v- 
coordinate  values,  if  the  slope  was  greater  than  1.0.  This  endpoint  procedure 
was  used  because  small  differences  in  x-values  can  lead  to  large  differences 
in  y-values  for  nearly  vertical  lines  and  similarly  for  nearly  horizontal 
lines . 

For  the  case  of  complete  domination,  shown  in  Figure  2.15  (c) ,  the  line 
segment  Vj  would  be  completely  discarded  in  favor  of  v^.  That  is,  the  new 
line  segment  would  be  the  old  v^. 

2.7.2  Pairing  Line  Segments 

The  goal  of  the  second  step  of  intersection  classification  and  detec¬ 
tion  is  to  pair  all  possible  line  segments.  This  step  would  associate  the 
two  sides  of  a  road,  for  example,  as  a  single  entity.  The  pairing  was  ac¬ 
complished  by  comparing  each  pair  of  line  segments  (v^v^)  such  that  (see 
Figure  2.17): 

(1)  The  difference  between  the  polar  angles  of  the  two  line  segments 
had  to  be  within  180°  +  A0  tolerance, 

(2)  the  distance  between  the  endpoints  and  the  other  line  had  to  be 
less  than  As^, 

(3)  the  line  segments  had  to  overlap  at  least  by  Ao,  and 

(4)  the  line  segments  did  not  intersect. 

After  this  process,  it  is  possible  to  have  the  situation  shown  in 
Figure  2.18,  where  Vj  and  Vj*  were  not  merged  because  they  were  too  far  apart, 
but  because  of  their  pairing  with  they  should  be  merged.  This  last  step 
was  introduced  because  the  intersection  of  two  roads  would  be  represented  by 
the  intersection  of  two  pairs  of  line  segments  -  making  detection  easier. 


Figure 


F igure 


.17.  In  order  for  and  Vj  to  be  paired,  the  distance 
a,  b,  c,  and  d  must  be  less  than  the  supplied 
tolerance  AS3. 


.18.  A  situation  where  v,  and  v j »  should  be  merged  and 
the  result  paired  with  . 
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Referring  to  Figure  2.18,  the  conditions  on  merging  v.  and  Vj ,  were: 

(1)  the  difference  in  distances  d2  and  dj  had  to  be  within  As^, 

(2)  the  distance  dj  must  be  less  than  2*max(d2 ,d -j) *Asj ,  and 

(3)  the  line  segments  Vj  and  \)j  i  have  to  be  on  the  same  side  of  . 

Pairs  were  typed  as  dark  lines  on  a  light  background  or  light  lines  on  a 
dark  background. 

2.7.3  Detecting  and  Classifying  Intersections 

The  last  step  is  to  detect  and  classify  intersections  according  to 
Figure  2.11.  This  step  is  accomplished  by  treating  each  pair  of  line  segments 
or  an  unpaired  line  segement  as  one  unit  and  compairing  all  such  units. 

For  each  combination  of  units: 

(1)  determine  if  they  intersect  or  would  intersect  if  their  lengths 
were  extended  by  As]_  at  each  end,  and  the  angle  between  the  inter¬ 
secting  units  is  not  within  0°  +  A0. 

(2)  if  they  pass  (1),  then  the  intersection  is  typed  as  1,2,  or  3 
depending  on  where  the  intersection  occurred.  If  the  inter¬ 
section  occurred  within  As^  of  the  ends  of  both  units,  then  the  type 
is  1;  if  it  occurred  within  As^  of  the  end  of  one  unit,  the  the  type 
is  2;  else  it  is  type  3.  Types  1,  2,  and  3  correspond  to  L,  T, 
and  X  intersections,  respectively. 

(3)  Finally  if  they  satisfied  (1)  and  (2),  the  intersections  are  finally 
classified  as  L  ,  l^,  etc.  depending  on  how  many  line  segments  are 
in  the  units.  For  example  if  two  pairs  are  involved  in  a  type 

1  intersection  then  the  intersection  is  classifies  as  I^. 
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2.7.4  Experiments  on  Detection  and  Classification  of  Intersections 

The  three  steps  of  detecting  and  classifying  intersections  were  applied 
to  a  series  of  sets  of  extracted  line  segments  from  images  of  hinges  and 
carburetor  covers.  These  images  were  selected  both  because  the  edges  were 
already  available,  and  they  contained  many  straight  edges.  The  images  used 
are  shown  in  Figure  2.19.  The  sets  of  extracted  edges  are  shown  before  and 
after  merging  in  Figures  2.20-2.24.  The  lists  of  paired  line  segments  and 
their  types,  and  the  detected  intersections  with  their  types  are  also  shown. 
The  detected  intersections  are  circled  on  the  images.  The  values  used  for 
the  tolerances  As^,  As2>  As^,  A0,  and  Ao  were  7  pixels,  7  pixels,  25 
pixels,  .75  radians,  and  50%  respectively.  Figure  2.25  shows  a  test  data 
set  demonstrating  the  capability  of  the  algorithm. 

The  results  shown  indicate  that  intersections  can  be  detected  and  clas¬ 
sified  automatically.  This  algorithm  can  easily  be  combined  with  EDGEY  to 
include  exploration  for  missing  line  segments  in  the  image.  Automatic  detec¬ 
tion  and  classification  of  intersections  are  important  steps  in  the  LNK 
registration  procedure  described  in  the  next  section. 
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Edges 

Type 
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17 

Dark  on  Light 

24 

25 

Dark  on  Light 

24 

27 

Light  on  DarK 

Figure  2.21  (a)  Original  edge  segments  extracted  from  image  in 

Figure  2.19(b).  (b)  Result  of  applying  merging 

algorithm.  Detected  Intersections  are  circled. 
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Figure  2.22  (a)  Original  edge  segments  extracted  from  image  in  Figure 

2.19(c).  (b)  Result  of  applying  merging  algorithm.  De¬ 

tected  Intersections  are  circled 
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Figure  2.23  (a)  Original  edge  segments  extracted  from  image  in  Figure 

2.19(d).  (b)  Result  of  applying  merging  algorithm.  De¬ 

tected  intersections  are  circled. 
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Figure  2.24  (a)  Original  edge  segments  extracted  from  image  in 

figure  2.19(c).  (b)  Result  of  applying  merging 

algorithm.  Detected  intersections  are  circled. 


(b) 


Figure  2.25  Test  image  showing  some  of  the  capabilities  of  the 
algorithm.  The  original  input  set  is  shown  in  (a) 
The  result  of  the  merging  step  is  shown  in  (b) . 
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2. 8  Region  Segmentation 

A  segmentation  of  an  Image  Is  a  partition  of  the  Image  into  disjoint 
subsets  whose  union  Is  the  entire  image.  For  surveys  of  segmentation  see 
[Riseman  and  Arbib  1977,  Kanade  1980,  Zucker  1976,  Pavlidis  1977].  Many 
segmentation  procedures  produce  an  initial  segmentation  and  then  apply  an 
iterative  procedure  such  as  merging  or  splitting  to  obtain  an  improved  seg¬ 
mentation.  Two  basic  approaches  to  finding  an  initial  segmentation  are  first 
locating  boundaries  or  first  locating  pixels  with  similar  feature  values. 

Boundary  detection  generally  consists  of  edge  detection  followed  by 
linking  of  edges  into  closed  boundaries  as  discussed  in  Section  2.2.  Edge 
detection  procedures  which  form  closed  edges  such  as  the  Marr  detector  [Marr 
1979]  and  relaxation  labelling  [Zucker  1977]  provide  a  segmentation  direct¬ 
ly.  Edge  detection  in  textured  images  is  a  difficult  task  which  depends  heav¬ 
ily  on  the  relative  scale  of  the  texture  and  the  regions'  sizes. 

Construction  of  regions  based  on  similarity  of  pixels  or  pixel  neighbor¬ 
hoods  requires  feature  extraction.  Features  commonly  measured  include  average 
gray  level  in  a  neighborhood,  variance,  average  edge  content  per  unit  area, 
average  orientation  of  local  edges  and  average  spot  size  of  uniform  contiguous 
areas.  Texture  measures,  such  as  co-occurrence  matrices  and  Fourier  transform 
ring  data  can  be  used  as  feature  measures  for  larger  areas.  Threshold  tech¬ 
niques  [Pavlidis  1977,  Price  1976]  can  be  used  with  these  features  to  provide 
an  initial  segmentation. 

Milgram  [1978]  describes  a  procedure  for  region  construction  using  evi¬ 
dence  from  several  sources  such  as  edge  information  and  pixel  feature  values. 
The  algorithm  selects  the  contours  at  different  thresholds  according  to  the 
support  of  the  edge  data  along  the  contours.  Zucker  [  1979  ]  gives  a  relaxa¬ 

tion  technique  for  constructing  regions  from  primitive  edges.  This  scheme 


60 


allows  points  to  be  considered  as  interior  or  boundary  points  of  a  region. 
The  relaxation  process  allows  edge  segments  separating  regions  to  prosper 


as  region  points  and  edge  points  reinforce  themselves. 

Pavlidis  describes  a  general  class  of  split-merge  algorithms  using  the 
notion  of  region  adjacency.  Given  a  criterion  for  deciding  if  a  single  region 
should  be  split  and  a  criterion  for  deciding  whether  two  adjacent  regions 
should  be  merged,  the  procedure  is  as  follows: 

1)  Split  each  region  which  should  be  split  according  to  the  splitting 
criterion.  Continue  this  until  no  further  regions  satisfy  the  split¬ 
ting  criterion. 

2)  If  step  3  has  not  been  executed  yet  then  keep  going,  else  if  no 
merges  occurred  in  the  last  execution  of  step  3,  then  stop. 

3)  Merge  any  two  adjacent  regions  which  should  be  merged  according  to 
the  merging  criterion.  Continue  this  step  until  no  adjacent  regions 
satisfy  the  merging  criterion. 
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The  concept  of  registration  is  crucial  to  image  interpretation. 
Very  generally  it  applies  to  matching  images  from  different  sensors, 
matching  images  taken  at  different  times,  or  matching  an  image  to  a  map 
(GDB) .  Registration  of  images  to  maps  is  viewed  here  as  an  important  step 
in  unlocking  rich  a  priori  knowledge  stored  in  the  data  base  for  use  in 
image  analysis.  To  succeed,  registration  clearly  must  be  achievable  with 
only  a  partial  primitive  image  analysis.  Once  registration  is  accomplished, 
more  complete  image  analysis  can  be  performed  under  map  guidance  and  hope¬ 
fully  will  be  much  more  reliable  and  efficient  than  image  analysis  without 
a  priori  information. 

Reasonable  human  and  automatic  registration  procedures  are  based 
on  "pass  points".  Pass  points  are  uniquely  identifiable  points  in  the 
image  usually  defined  by  special  edge  context,  i.e.  points  where  rivers  or 
roads  intersect,  corners  of  buildings,  mountain  peaks,  etc.  Humans  will 
uniquely  identify  pass  points  because  ambiguity  can  be  removed  by  their 
large  amount  of  global  knowledge.  Automatic  procedures,  on  the  other  hand, 
typically  work  with  far  less  global  knowledge  and  must  be  built  to  tolerate 
ambiguities  in  matching  single  pass  points  from  the  image  with  those  of  the 
map.  The  next  section  gives  a  brief  survey  of  some  registration  work  which 
is  used  to  set  the  stage  for  introducing  the  L.N.K.  registration  procedure 
developed  in  Section  3.2. 
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Survey  of  previous  registration  work 

Mathematically,  we  formulate  the  registration  problem  as  the 
problem  of  determining  a  transformation  Ta  that  maps  a  point  =  (x,y) 
in  the  first  image  space  to  "corresponding"  point  P2  =  (u,v)  in  the 
second  image  space.  A  definition  of  "corresponding"  can  be  subtle. 
Points  can  correspond  because  of  their  obvious  and  unambiguous  structure 
and/or  interpretation,  such  as  the  tip  of  the  Washington  Monument  as 
seen  in  two  pictures.  Points  can  also  correspond  by  the  relationship 
which  Ta  places  on  them;  —  i.e.  a  point  (x,y)  on  the  Mall  lawn,  in 
image  one,  corresponds  to  a  point  (u,v)  in  image  two  because,  Tu(x,y)  = 
(u,v).  The  first  type  of  point  will  be  called  a  pass  point  or  control 
point.  Pass  points  can  be  used  to  construct  Ta  which  can  then  be  used 
to  relate  all  other  points  in  the  image  domain  regardless  of  any  obvious 
structural  correspondence  between  the  points.  In  general,  we  are  less 
certain  about  the  correspondence  of  points  on  the  Mall  lawn  than  we 
are  about  the  Monument  tips. 

A  straightforward  and  common  registration  technique  is  to 


use  human  selection  of  pass  points,  in  the  two  images,  resulting  in 
a  set  of  corresponding  points  C  ={(p11>P2i^»  ^12,P22^’  ■••> 

!  •  The  desired  transformation  T^  may  have  from  two  to  six 
parameters  depending  on  the  narticular  model  of  image  formation.  The  best 


transformation  Ta  can  be  defined  as  that  Ta 

e(  a  )  =  d2(T  (P  )P  ) 

V  ‘  i=l ,k  U  a  v  li^  2i' 


,  such  that 


is  minimized,  where  d  is  the  distance  between  the 


control  point  P^^ 
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in  the  second  image,  and  the  transform  of  the  corresponds  _  point  P  from 

the  first  image.  Classical  least  squares  procedures  can  be  used 

to  determine  Ta  [  V an  Wte  1977  ].  To  automate  the  registration  procedure 
a  method  of  automatic  selection  of  the  points  in  SetC  must  be  devised. 

The  least  squares  fit  of  T^  will  be  sensitive  to  any  errors  in  this  set 
of  corresponding  points. 

Horn  and  Bachman  [1977]  have  a  procedure  which  uses  all 
points  in  the  image  overlap  and  hence  requires  no  feature  selection  to 
get  Set  C.  However,  the  computation  of  e(  a)  becomes  involved  and  hill¬ 
climbing  from  an  approximate  ^  must  be  used.  Sometimes  such  an 
approximation  is  available  from  knowledge  of  the  attitude  of  the  sensing 
platform. 

Barrow  et  al  [1977]  proposed  using  only  salient  edge  points  for 
set  C  and  described  a  computationally  fast  method  of  computing  e(  a  ) 
called  "chamfer  matching".  Their  procedure  also  used  hill-climbing 
from  an  approximate  a  to  get  a  general  6-parameter  transformation  TQ, 
valid  for  even  the  modeling  of  oblique  imagery. 

The  heart  of  the  registration  problem  is  the  selection  of  the 
set  of  control  points  C.  In  map  making  this  is  usually  a  human  operation. 

Here  we  are  exploring  automatic  techniques.  If  an  approximate  Ta  is 
available  it  is  possible  to  search  a  limited  area  of  an  image  to  find 
a  point  corresponding  (in  structure)  to  a  point  in  a  map  (second  image). 

Block  correlation  can  be  used  so  long  as  T  effectively  removes  rotation. 

Van  Wie  and  Stein  [1977]  report  some  success  at  this  for  repeat  ERTS 
coverage.  However,  even  use  of  gradient  images  for  block  correlation  was 


not  always  reliable.  Deficiencies  in  block  correlation  have  also  been 


pointed  out  by  Crombie  [1975].  L.N.K.  believes  in  the  following  two 
principles  which  have  tempered  the  research  work  reported  below. 

(1)  Correlation  for  control  point  matching  should  consider  higher 
level  structure  in  the  neighborhood  defining  the  control  point.  For 
instance,  edges  or  lines  should  be  detected  and  their  points  of 
intersection  typed  as  "T's"  or  "X's"  before  correspondences  are 
attempted. 

(2)  Correspondences  drawn  from  local  matching  criteria  must  be 
tempered  via  feedback  from  global  registration  knowledge.  For 
instance,  many  crossroads  will  individually  look  alike,  but  a 
globally  valid  registration  transformation  can  pair  them  unambigu¬ 
ously,  using  the  information  contained  in  the  spatial  distribution 
of  the  entire  set. 

L.N.K.  has  developed  a  registration  procedure  which  integrates  both 
local  and  global  matching  criteria  and  has  been  quite  successful  in  ac¬ 
counting  for  rotations  and  translations  on  vertical  photography.  The 
next  section  gives  the  formal  development  of  the  procedure  and  experi¬ 
ments  on  real  data  are  reported  in  Section  3.3. 


3.2 


L.N.K.  Registration  Procedure 

A  technique  has  been  developed  by  L.N.K.  Corporation  for  auto¬ 
matically  registering  vertical  photography  with  maps  or  other  vertical  t 

t 

photography.  This  technique  has  been  successfully  demonstrated  at  L.N.K. 
and  on  the  ROSA  system  at  the  Research  Institute  of  USAETL.  The  current 
assumptions  (limitations)  are  that  the  registration  transformation  be 
limited  to  an  arbitrary  rotation  0  and  a  translation  (Ax, Ay)  which  is  no 
larger  than  half  of  the  image  diameter.  Due  to  relief  displacement  and 
distortion,  the  derived  Ta  is  only  an  approximate  transformation.  How¬ 
ever,  Ta  could  be  refined  as  will  be  shown.  In  addition,  by  first  deriv¬ 
ing  a  global  approximation  it  is  likely  that  a  more  robust  and  efficient 
procedure  is  obtained.  A  method  of  introducing  local  adjustments  to  Ta 
for  matching  individual  features  is  covered  in  Section  5. 


3.2.1  The  General  Procedure 

Registration  is  viewed  as  a  three  step  procedure.  Here  the  ter¬ 
minology  pertains  to  matching  image  structure  to  a  map  or  model.  However, 
in  fact,  the  map  or  model  may  be  replaced  by  an  image  of  the  same  or  dif¬ 
ferent  type  as  has  been  done  in  several  experiments. 


(Step  Gl:  feature  extraction  and  local  matching) 

Obtain  corresponding  structures  in  the  image  and  map.  Struc¬ 
tures  correspond  when  they  have  the  same  shape,  size,  color,  etc. 

The  correspondences  are  plausible  from  local  evidence  but  may  be 
ambiguous  or  incorrect.  In  fact,  each  image  structure  may  be  paired 
with  one,  none,  or  many  map  structures. 

(Step  G2:  global  interpretation  via  T  ) 

Determine  transformation  parameters  a  =  (a^a  ,  .  .  .  ,a  ) 
such  that  Ta  maps  at  least  some  image  structures  (^i.e.  points, 
lines,  arcs,  etc.)  onto  corresponding  map  structures. 
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(Step  G3:  global  match  evaluation) 

Determine  the  global  goodness  of  match  Induced  by 
image  structures  and  map  structures.  (One  way  to  do  this 
compute  the  mean  squared  distances  between  map  structures 
image  structures  transformed  by  Ta> ) 


on  the 
is  to 
and 
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3.2.2 


The  specific  L.N.K.  registration  technique  is  a  hybrid  of  tempi.) 
matching  and  structural  analysis  and  combines  the  advantages  of  those  two 
procedures.  The  specific  interpretation  of  the  general  steps  above  are  as 
follows. 


(SI) 


(S2) 


(S3) 


Assume  all  structures  of  the  same  type  correspond.  For  example, 

assume  each  straight  line  segment  in  the  image  can  correspond 

to  each  straight  line  segment  of  the  model,  each  convex  curve 

in  the  image  can  correspond  to  each  convex  curve  of  the  model, 

etc.  For  each  pair  of  structures  (s.  ,s  ),  where  s  and  s  are 

i  m  i  m 

structures  from  the  image  and  map  respectively,  compute 
transformation  parameters  a  and  place  a  unit  of  measure 
in  a-  parameter  space. 

Possible  transformations  T  between  image  and  model  are 

a 

detected  as  clusters  in  a-  parameter  space  formed  in  step 

SI  because  heavy  measure  at  a  in  a-  space  means  that 

o 

many  correspondences  are  explained  by  T 

ao 

Evaluation  of  the  match  strength  of  each  from  step  S2 
is  obtained  by  either  computing  an  average  distance  between  all 
corresponding  structures  or  by  counting  the  number  of  image 
structures  explained  by  the  model  structures  under 


3.2.3  A  Simple  Example 

A  simple  example  of  this  process  is  illustrated  in  Figure 
3.1.  Assume  that  the  image  can  be  represented  by  the  4  directed  edge 
elements  shown  in  (a)  while  the  map  contains  the  edge  elements  in  (b). 


68 


Figure  3.1  Example  of  global  registration  via  clustering  of 
local  evidence.  Image  edge  elements  in  (a)  need 
to  be  rotated  45°  and  then  translated  (4.5,  -2.) 
to  be  transformed  into  corresponding  map  edge  ele¬ 
ments  in  (b).  (c)  16  units  of  measure  are  amassed 

in  (0,xs,ys)-space  forming  a  cluster  at  (0=0.79, 
xs=4 . 5,ys=-2.0) . 


It  is  assumed  that  the  length  of  the  edge  elements  is  accurately  known. 
There  are  16  possible  ways  that  an  edge  element  from  (a)  can  be  paired  with 
an  edge  element  in  (b).  Each  pairing  yields  unique  transformation  para¬ 
meters  (0,xs,ys)  as  shown  in  (c).  Four  of  the  16  possible  pairings  yield 
a  consistent  interpretation  —  rotate  by  0  =  0.79  radians  and  translate 
by  (4.5,  -2.0).  The  parameters  from  the  4  correct  pairings  form  a  cluster 
in  a  =  (0,xs,ys)  -  space,  while  the  parameters  from  incorrect  pairings  are 
sparsely  distributed  in  the  space.  In  practical  cases  there  will  be  many 
more  than  4  primitive  structures  and  not  all  pairings  will  be  possible 
(i.e.  due  to  size  or  shape  differences)  so  the  presence  of  a  cluster  in 
the  parameter  space  should  be  even  more  obvious . 

3.2.4  A  More  Complex  Example 

The  clustering  phenomena  observed  above  should  always  occur, 
as  long  as  the  vectors  representing  the  image  and  model  have  many  correct 
matches  in  common.  Imperfect  automatic  feature  detection  or  imperfect 
objects  in  the  image  would  result  in  additional  points  in  the  cluster 
space  or  fewer  points  in  the  cluster  space,  but  the  character  of  the 
clusters  should  remain  the  same  despite  a  fair  amount  of  variation  in 
the  image  features.  This  has  been  confirmed  in  experiments  registering 
images  to  maps  or  other  images. 

Some  features  which  '  e  extracted  from  images  automatical¬ 
ly  include  straight  edge  S'  .  ..its,  omers,  points  of  high  curvature, 
and  regularly  shaped  spots  (i.e.  circles  or  rectangles).  Experiments 
reported  in  Section  3.3  show  that  the  probability  of  detecting 
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individual  features  can  be  as  low  as  20%  and  recognition  of  the  whole 
can  still  be  achieved  using  the  proposed  clustering  technique. 

Consider  the  features  of  terrain  observed  from  a  high  altitude 
above  Harrisburg,  Pennsylvania  (Figure  3.2  ).  Five  points  of  high 
curvature  and  six  points  of  intersection  of  lineal  features  are  identified 
in  Figure  3.2b.  For  instance,  point  I  ^  is  the  junction  between 
Route  15  and  the  Pennsylvania  Turnpike,  while  point  is  a  point  on 

Sherman  Creek.  The  30  vectors  shown  in  Figure  3.2b  are  formed  with  a 
tip  on  an  intersection  point  and  tail  on  a  point  of  high  curvature. 

While  the  vectors  of  Figure  3.2b  look  nothing  like  the  region  they 
represent,  they  are  very  useful  for  registration.  Figure  3.2c  shows 
10  vectors  formed  by  features  detected  in  an  image  taken  from  a  per¬ 
spective  145°  off  of  the  original  perspective  in  Figure 3. 2a  and 
3.2b.  The  feature  points  were  selected  independently  by  two  researchers 
with  the  same  instructions.  There  are  10x30  =  300  possible  matches  of 
vectors  from  3.2b  with  vectors  from  3.2c,  almost  all  of  which  are 
ignored  because  of  differences  in  length.  The  result  of  automatic  clus¬ 
tering  in  the  transformation  parameter  space  is  shown  in  Figure  3. 2d. 

The  correct  transformation  parameters  (  8,  Ax,  Ay)  =  (145° ,38 ,115) 
are  obtained  first  from  the  strongest  cluster  and  there  is  little  com¬ 
petition  from  other  clusters.  Under  the  best  transformation  all  10 
image  vectors  are  explained  by  the  set  of  model  vectors.  Within  the 
tolerance  allowed,  11  of  the  30  model  vectors  are  "seen  in  the  image". 

A  root -me an- square  match  weight  of  33.16%  can  be  assigned  to  the 
match  based  on  an  average  over  30  individual  matches. 
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Figure3.2(b)  5  points  of  curvature  and  6  points  of  intersection  detected 
by  researcher  #1  forming  30  abstract  edges  representing  the  scene  in  (a). 
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3.2.5  Details  of  the  L.N.K.  registration  procedure 

Recall  that  the  L.N.K.  registration  procedure  consists  of 
3  conceptual  steps.  First,  image  structures  and  map  structures  must 
be  paired  so  that  for  each  possible  corresponding  pair,  transformation 
parameters  a  (or  a  set  of  several  transformation  parameters)  can  be 
computed  which  transform  the  image  structure  onto  the  map  structure. 
Handling  all  such  pairs  populates  the  a-space  as  discussed  below. 

Second  ,  clusters  in  a-space  must  be  detected  so  that  those 
which  cause  a  large  amount  of  image/map  overlay  are  discovered. 

Third  t  a  quantitative  measure  of  the  amount  of  overlay  is  computed 
so  that  a  transformation  T can  be  evaluated  for  acceptance,  relative  to 
some  threshold  criteria  or  relative  to  another  possible  Ta  . 

Before  proceeding  it  is  essential  to  emphasize  that  the 
registration  procedure  is  edge-based  and  can  operate  on  either  real  or 
abstract  edges.  Real  edges  correspond  to  the  location  of  real  contrast 
changes  as  indicated  in  the  gradient  image.  The  direction  of  the  edge 
indicates  in  which  direction  tonal  increases  occur.  The  length  of  a  real 
edge  in  a  map  may  be  precise  but  it  certainly  is  not  precise  in  an  edge 
element  automatically  extracted  from  imagery.  Usually  image  edge  elements 
represent  some  small  segment  of  the  map  edge  and  in  the  experiments 
reported  here  are  usually  only  1/8  inch  long.  Abstract  edges  are  formed 
by  joining  two  arbitrary  point  features.  The  direction  is  arbitrary  but 
fixed  and  length  is  fairly  precise  because  the  endpoints  are  detectable 
pass  points.  As  shown  in  the  experiments,  although  both  kinds  of  edges 
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can  be  used,  the  abstract  edges  produce  cleaner  clustering  and  more  precise 
transformations.  The  major  reason  is  that  pairs  of  corresponding  abstract 
structures  can  be  restricted  to  only  those  that  agree  very  closely  in  length 
and  exactly  in  end  point  type. 

3. 2. 5.1  Pairing  edge  elements  and  populating  q-space 

We  assume  that  a  specifies  a  rotation  and  translation:  a  = 
(0,Ax,Ay).  if  the  orientation  of  the  image  edge  element  (EE)  is  6^  and 
the  orientation  of  the  map  EE  is  0m  then  it  is  easy  to  compute  the  rota¬ 
tional  part  of  that  a  that  will  overlay  the  two:  a  =0^  -  0^.  Referring 
to  Figure  3.3  it  is  easily  seen  that  Ax  and  Ay  are  then  linearly 
constrained  by  the  following  equation. 

Ax  cos6m  +  Ay  sinam  +  (r^  -  r  )  =  0 

Actually  the  rotated  EE  A'B'  is  not  free  to  lie  anywhere  along 
the  line  determined  by  points  C  and  D  but  must  lie  along  the  segment  C'D'. 
Thus  the  linear  equation  above  can  be  replaced  by  two  points  between 
which  (Ax, Ay)  must  lie.  The  first  point  is  easily  determined  by  that 
(Ax^.Ay^)  necessary  to  translate  point  A'  onto  point  C.  Similarly  the 
other  extreme  is  determined  by  that  (Ax£,Ay2)  necessary  to  translate 
point  B'  onto  point  D.  In  practice,  we  allow  for  some  overshoot  (  dto]  ) 
of  points  C  and  D  because  the  Hough  detector  may  in  fact  overshoot  the 
real  edge  somewhat  when  EE  AB  is  detected  and  because  of  error  in  0  . 

Errors  of  2°  in  9^  are  quite  common  for  the  32x32  window  used. 

Thus  for  each  pair  of  corresponding  EE's  the  potential  regis¬ 
tration  transformations  are  triples  a=(0,Ax,Ay)  such  that  0  =  0-0.+O 

m  i  error 
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Image  edge  AB  or  (Q^.r^)  is  rotated  -  0^  into  A'B'  or 

(9  ,rj  to  be  parallel  to  map  edge  CD  or  (9  ,r  ).  Unit  vector  from  A' 
ml  mm 

to  R  is  (-cos9  ,-sin6  ).  Projection  of  A'C  onto  A'R  has  constant  length 
m  m 

for  all  C  on  line  yielding  following  relation  between  At  and  Ay. 

A'C  •  (-cos9  ,-sin9  )  =  r.-r 
m  m  i  m 

(Ax, Ay)  •  (-cos0  ,-sin0  )  =  r,  -  r 
m  m  l  m 

Ax  cos9m  +  Ay  sin0  +  (r.  -  r  )  =  0 
m  m  i  m 

More  3imply,  (Ax, Ay)  lies  on  a  line  segment  between  the  two  points  (Ax^.Ay^) 

(Cx' ,Cy')  -  (Ax' , Ay')  and  (Ax2>Ay^)  =  (Dx',Dy')  -  (Bx',By')  assuming  that 

map  edge  CD  is  at  least  as  long  as  image  edge  AB. 

Figure  3.3  Derivation  of  constraints  on  a  =  (0,Ax,Ay) 
for  match  of  image  edge  element  and 

map  edge  element  (9  ,r  ) . 


and  (Ax, Ay)  lies  between  (Ax^.Ay^) (AX2 , Ay£) .  The  error  In  Ax, Ay  Induced  by 

©error  *s  c^e  order  r®error‘  centering  the  origin  for  rotation  in 

the  window,  the  effect  of  this  error  is  minimized.  Assuming  that  r=2000 

stagels  and  that  0  <2  degrees,  the  error  induced  in  (Ax, Ay)  by  9 

error-  erroi 

is  at  most  2000  stagels  x  2  degrees/360  degrees  x  2tt  =  60  stagels.  Due  to 
the  threshold  used  for  straight  line  detection,  the  Hough  detector  could 
overshoot  the  end  of  an  edge  by  half  of  the  window  size  or  128  stagels. 

By  allowing  another  60  stagels  for  error  we  get  an  approximation  of  188 
stagels  for  DTOL.  Thus  by  theoretical  considerations  alone,  we  arrive  at 
clustering  parameters  ANGT0L=5  degrees,  STOL=50  stagels,  and  DTOL=188 
stagels.  Most  of  the  experiments  reported  below  were  performed,  before 
this  theoretical  analysis,  with  ANGT0L=5  degrees,  ST0L=50  stagels  and 
DTOL=200  stagels  yielding  good  results. 

With  a  so  constrained  there  may  be  theoretically  an  infinite 
number  of  points  to  contribute  to  a-space.  However,  the  inherent  error 
on  0,  Ax,  and  Ay  allows  a  discretization  on  a-space  which  makes  cluster¬ 
ing  by  binning  a  good  technique.  For  each  pair  of  corresponding  structures 
we  place  the  5-tuple  (0, Ax^jAy^jAx^^y^)  in  our  cluster  space.  We  succes¬ 
sively  zoom  in  on  clusters  by  redefining  a  grid  of  bins  and  for  each  level 
of  clustering  the  set  of  5-tuples  is  examined  for  incrementing  the  bins. 
Whenever  points  (Ax^,Ay^)  and  are  in  separate  bins,  more  than 

one  bin  count  is  incremented. 
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3. 2. 5. 2  Clustering  in  a-space 

In  clustering  by  binning  we  work  with  a  fixed  number  of  bins.  If 

tin.-  0 ,  Ax,  and  Ay  dimensions  are  quantized  into  10  units  then  there  must 

be  10x10x10  bins.  (Actually  there  will  be  more  as  explained  later.) 

The  "size"  of  each  bin  is  determined  by  the  range  of  the  variable  divided 

by  10.  The  range  of  the  variable  depends  on  the  level  of  clustering. 

For  instance,  at  the  first  level  the  range  atf  0  is  360°  so  each  bin 

represents36°  along  the  6  dimension.  We  always  assume  that  Ax  and  Ay 

can  vary  no  more  than  half  of  the  diameter  of  the  image  so  for  a  8000  x 

8000  stagel  image  the  range  of  Ax  and  Ay  is  4000  and  the  size  of  each 

bin  is  then  36°  x  400  stagels  x  400  stagels  at  level  one.  When  a 

level  one  bin  is  found  to  be  "heavy"  we  can  zoom  in  by  resetting  the 

full  ranges  of  0,  Ax,  and  Ay  to  be  those  taken  on  for  the  heavy  bin. 

Thus  at  level  two  the  size  of  a  bin  is  3.6°  x  40  stagels  x  40  stagles 

and  by  level  three  it  is  0.36°  x  4  stagels  x  4  stagels.  Clearly  three 

levels  of  clustering  will  be  too  much  for  typical  problems.  Clustering 

should  proceed  only  until  the  bin  size  is  comparable  to  the  error 

inherent  in  determining  6 ,Ax  and  Ay.  At  any  level,  a  detection  is  only 

made  when  an  acceptable  number  of  points  lie  in  the  bin.  The  acceptance 

threshold  will  be  dependent  upon  the  reliability  of  the  edge  or  pass  point 

detectors.  If  we  demand  that  half  of  the  detected  edge  elements  should 

fall  on  edges  existing  in  the  map  then  our  cluster  detection  threshold 

should  be  0.5N  where  N  is  the  number  of  detected  edge  elements.  In 
e  e 

the  experiments  documented  in  Section  3.3  no  absolute  cluster  threshold 
was  applied.  Instead,  up  to  3  of  the  best  clusters  at  up  to  3  levels 
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were  pursued  and  the  corresponding  transformations  were  evaluated 
by  a  technique  described  in  Section  3. 2. 5. 3. 

Clustering  by  binning  can  suffer  severely  from  boundary 
effects  as  illustrated  in  Figure  3.4.  A  cluster  actually  present 
can  be  missed  due  to  the  quantization  of  the  ci-space.  To  avoid  the 
distribution  of  points  from  a  cluster  to  several  neighboring  bins,  3 
other  overlapping  or  offset  seta  of  bins  can  be  used.  The  bins  of 
the  other  grids  are  displaced  by  1/2  of  the  bin  size  relative  to  the 
first  grid  of  bins  (See  Figure  3.4).  Suppose  that  a  cluster  of  N 
points  exists  in  a-space  and  has  diameter  d.  With  the  above  technique 
there  is  guaranteed  detection  at  threshold  N  so  long  as  the  bin  size  is 
at  least  2d.  If  bin  size  below 2d  is  used,  the  cluster  will  be  broken 
up  and  detection  of  T^  will  be  jeopardized,  d  is  determined  from  the 
error  inherent  in  the  parameters  of  the  space.  As  previously  stated,  the 
error  is  not  only  due  to  measurement  error  but  also  contains  approximation 
error  since  we  are  using  only  a  rotation  and  translation  whereas  the 
true  T^  may  have  nonlinear  effects.  In  the  case  of  registration  of 
stereo  images  differential  displacements  will  occur  which  will  be  a 
function  of  elevation  off  some  base  level.  The  component  of  error 
due  to  elevation  is  bounded  by  considering  the  maximum  relief  to  be 
observed  in  the  imagery. 
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Figure  3.4  Clustering  using  M-  sets  of  overlapping  bins. 

No  bin  in  first  grid  contains  more  than  4 
points  but  15  points  enter  bin  a'bCD*  in  one 
offset  grid. 
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3. 2. 5. 3  Evaluating  the  goodness  of  fit  for  T 


For  matching  point  patterns  the  root-mean-square  (RMS)  distance 
between  map  points  and  transformed  image  points  is  a  good  measure  of 
the  match.  For  abstract  edges  the  RMS  technique  is  also  applicable 
since  two  precise  points  determine  each  abstract  edge.  However,  for  real 
edges  the  point  concept  is  not  useful  since  it  is  unlikely  that  the  whole 
of  the  image  edge  is  present.  A  heuristic  RMS  distance  computation  is 

described  below  which  was  used  to  evaluate  each  candidate  T  in  the 

a 

reported  registration  experiments. 

Given  a  candidate  Ta  determined  by  ct  =  (  6,Ax,Ay)  we  can 
heuristically  evaluate  how  good  image  edge  (xl,yl)  -  (x2,y2)  overlays  map 
edge  (ux,uy)  -  (vx,vy)  under  Ta  .  Refer  to  Figure  3.5.  A  function  is 
constructed  which  measures  how  well  TQ  aligns  the  edges  in  the  map 
space  in  terms  of  direction  and  proximity.  Let  (TX1,TY1)  -  (TX2.TY2) 
be  the  image  edge  transformed  under  Ta  .  If  this  transformed  edge 
has  direction  which  differs  from  the  map  edge  (ux,uy)  -(vx,vy)  by  more 
than  the  angle  tolerance  ANGTOL  then  the  match  weight  is  0  (EMATCH  (Ta  , 
(xl,yl) (x2 ,y2) , (ux,uy) (vx,vy) )  =  0).  ANGTOL  should  be  set  according 
to  the  9  error  in  Ta  .  Most  of  the  possible  pairings  of  image  and  map 
edges  will  have  EMATCH  of  0  by  failing  this  test.  If  the  transformed 
image  edge  has  direction  compatible  with  the  map  edge  then  the  EMATCH 
value  is  computed  as  the  product  of  a  length  overlap  and  a  proximity  value. 

To  get  the  length  overlap  LMATCH,  the  transformed  edge  is  prolected  onto  the 
map  edge  and  any  LEFT  or  RIGHT  overshoots  are  measured  and  adversely 
affect  the  LMATCH  as  shown  in  Figure  3.5.  Note  that  the  LMATCH  can 
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Figure  3.5 


Computation  of  a  match  score  between  a  map  edge 
and  an  image  edge  under  transformation  Ta. 


be  1.0  even  If  the  image  edge  is  much  shorter  than  the  map  edge.  A 
proximity  value  or  DMATCH  is  computed  by  comparing  the  average  distance 
between  points  on  the  edges  and  a  distance  tolerance  DTOL.  DTOL  should 
be  set  from  the  error  inherent  in  Ax  and  Ay. 

Edge  pairs  which  have  EMATCH  =  0  are  assumed  not  to  correspond 
to  the  same  image  structure  while  those  pairs  with  EMATCH  ^  0  are  assumed 
to  be  corresponding.  Theoretically  each  abstract  or  real  image  edge 
should  correspond  with  at  most  one  map  edge  while  each  real  map  edge  may 
correspond  to  many  real  edges  (short  segments  of  the  same  edge) .  The 
L.N.K.  registration  software  records  all  correspondences  made  by  using 
the  EMATCH  function  on  all  possible  edge  pairs.  A  "match  matrix"  may  be 
optionally  output  when  T  is  evaluated.  Not  only  does  the  match  matrix 
show  the  goodness  of  T^  but  it  also  shows  where  verification  must  be  done 
to  locate  missing  image  edges  or  where  positive  change  might  have  been 
detected  due  to  image  edges  without  matches  in  the  map. 
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3. 3  Registration  Experiments 


This  section  describes  some  experiments  on  registering  aerial  imagery. 

In  each  case  the  "true"  transformation  is  provided,  for  comparison  purposes 
This  transformation  was  obtained  by  calculating  the  transformations  that 
aligned  known  corresponding  abstract  edges.  Due  to  distortions  present,  such 
as  those  caused  by  the  relief  of  the  terrain,  this  "true"  transformation  is 
only  approximate  and  ranges  on  the  variability  have  been  estimated  and  provided. 
This  fact  should  be  taken  into  account  when  evaluating  the  registration  results. 

3.3.1  4621  Image 

The  first  set  of  experiments  used  the  image  designated  "4621"  and  shown  in 
Figure  1.1.  In  order  to  determine  the  "true"  transformation,  registration  was 
performed  using  abstract  edges  from  the  image  and  its  map.  The  abstract  edges 
were  formed  by  connecting  intersection  points  in  the  following  way: 

(1)  L-intersection  points  to  Y-  or  T-intersection  points, 

(2)  L-intersection  points  to  X-intersection  points, 

(3)  Y-  or  T-intersection  points  to  X-intersection  points. 

The  intersection  points  for  the  image  were  obtained  using  an  interactive 
s  ning  program  under  human  guidance.  The  abstract  edges  were  then  con¬ 
structed  by  hand,  and  the  result  is  shown  in  Figure  3.6. 

A  map  of  4621  was  created  by  using  a  Talos  digitizer  to  extract  a  set 
of  pass  points.  Most  of  these  pass  points  were  intersection  points  and  the 
entire  set  is  listed  in  Table  3.1.  A  set  of  abstract  edges  was  constructed 
using  the  same  criteria  used  for  the  image  abstract  edges,  and  the  result  is 
presented  in  Figure  3-7. 

The  registration  procedure  was  performed  on  the  two  sets  of  abstract 
edges  and  the  result  appears  in  Figure  3.8.  30  of  the  43  image  abstract  edges 
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Table  3-1  Digitization  of  points  on  image  4621  using  Talos  digitizer 
(0.001  inch  resolution)  and  scanning  stage  (0.0005  inch 


IMAGE  PT  # 

(see  Fig.  1.1) 

resolution)  . 

TALOS  * 

(photo) 

X  Y 

STAGE  * 
(transparency) 

X  Y 

DESCRIPTION  ** 

1 

2498 

3641 

5000 

5000 

X 

2 

4365 

2747 

8722 

6803 

T 

3 

5583 

2191 

11120 

8052 

X 

4 

3044 

4744 

3913 

7203 

T 

5 

3342 

5357 

3314 

8405 

T 

6 

3611 

5908 

2814 

9511 

T 

7 

1980 

6678 

- 

- 

T 

8 

3975 

5750 

3508 

9946 

T 

9 

5516 

5073 

6563 

11448 

T 

10 

7130 

3514 

10964 

12108 

Y 

11 

7430 

4091 

10367 

13304 

L 

12 

7723 

4700 

9866 

14600 

L 

13 

8642 

4258 

- 

- 

X 

14 

6952 

3223 

11209 

11461 

A 

15 

6942 

3613 

10565 

12000 

L 

16 

7233 

4181 

10010 

13101 

T 

17 

4861 

7575 

1710 

13600 

X 

18 

5980 

7667 

- 

- 

Y 

19 

6444 

7480 

- 

- 

T 

20 

9471 

6190 

- 

- 

T 

21 

5013 

7886 

1399 

14295 

A 

22 

4480 

7729 

- 

- 

T 

23 

7346 

9632 

- 

_ 

A 

24 

9282 

5763 

- 

- 

A 

25 

9700 

6104 

- 

- 

Y 

26 

5279 

7999 

1513 

14804 

A 

27 

4010 

1991 

9388 

5480 

A 

28 

5480 

2552 

- 

- 

L 

29 

4808 

2527 

- 

- 

A 

*  The  photo  in  Figure  1.1  was  rotated  roughly  50  degrees  clockwise  when 
mounted  on  the  digitizer.  The  corresponding  transparency  was  not 
rotated  when  mounted  on  the  stage. 

**  L,X,T,  and  Y  indicate  type  of  intersection,  while  A  indicates  an 
arbitrary  point  on  a  straight  road  segment. 
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43,1000 ,30  00,14000  , 160  0  0  , 
1056S , 12000 ,3508,9946, 
10565 ,12000,2814,9511 , 
10565,120  0  0  ,3314,84  05, 
10565,12000,3913,7203, 
1056S, 12000 ,8722 ,6803, 

10565.12000 . 1710 . 13600 , 
10565,12000  ,5000,5000, 

9866.14600.1710.13600, 
9866 ,14600  ,5000,5000, 

9866.14600.1 1120.8052, 

10367.13304 . 1710.13600, 

10367 ,13304,5000 ,5000  , 

10367 .13304.11120 .8052, 

10010. 13101 . 1710.13600, 


10010,13101  ,5000 ,5000 , 
10010,13101 ,11120 ,0052, 

10964.12108.1710.13600, 

10964.12108.5000.5000, 

10964 .12108 .11120.8052 , 

6563.11448.1710.13600, 

6563.11448.5000.5000, 

6563.11448.11120.8052, 

3508 .9946 .5000 .5000  , 

3508.9946.11120.8052, 

8814.9511 .5000.5000  , 

2814.9511.11120.8052, 
3314,8405,11120,8058, 

3913.7203.1710.13600, 
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were  matched  correctly  with  all  30  of  the  map  abstract  edges,  giving  the 
"true"  trans format  ion  (An=309o't  1  ° ,  Ax_598±60,  Ay=- 1  378  1  AO)  . 

3. 3. 1.1  Registration  using  real  edges. 

The  image  edges  were  found  using  the  lineal  feature  detection  process 
discussed  in  Section  2.  A  set  of  104  real  edges  were  found  and  are  shown  in 
Figure  2. 9a.  The  map  edges  were  constructed  by  connecting  intersection  points 
listed  in  Table  3-1  that  were  actually  connected  in  the  image.  This  set  is 
presented  in  Figure  3.9. 

The  results  of  registering  the  real  edges  appear  in  Figure  3.10.  The 
transformation  with  the  highest  weight  was  (307°, 539, -1371)  which  matched  63 
of  the  104  image  edges  with  9  of  the  18  map  edges.  The  difference  in  the 
transformation  parameters  from  the  "true"  values  would  lead  to  a  difference  in 
position  of  77  stagels  or  .039  inches  on  the  image  for  a  ooint  at  (1000,1000). 

3. 3. 1.2  Robustness  of  Registration  Procedure 

The  experiment  described  above  assumed  a  100%  reliable  feature  detector 
since  the  features  were  chosen  under  human  control.  Since  the  likelihood  of 
obtaining  this  degree  of  reliability  with  a  completely  automatic  feature  de¬ 
tector  is  low,  experiments  were  performed  on  abstract  edge  sets  with  varying 
degrees  of  reliability.  The  detection  probabilities  used  were  80%,  60%,  40%, 
and  20%.  Thus,  feature  points  were  chosen  with  a  probability  of  80%,  or  60%, 
etc.  and  abstract  edges  were  formed  using  the  same  criteria  as  before. 

The  80%-rellable  edge  set  was  created  by  using  a  table  of  random  numbers 
to  set  up  a  selection  process  such  that  each  feature  point  had  an  80%  probabi¬ 
lity  of  detection.  The  selection  process  used  was  to  look  at  a  sequence  of 
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(b) 

Figure  3.4  Listing  (a)  and  plot  (b)  of  real  map  edges 
from  4621 
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INPUT  5,50,200,3, 0,0 ,0,132,0,0, 
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Figure  3.10  Results  obtained  from  registration  software  when  run 

with  set  of  real  map  edges  and  real  image  edges  from  4621, 
EMATCH  =  MTCHWT  /  10000  =  .48. 


random  numbers  and  eliminate  those  numbers  ending  in  a  "I"  or  a  "2".  The 
remaining  sequence  numbers  were  then  used  to  select  the  feature  points  from 
the  original  set.  The  60%  and  other  sets  were  constructed  similarly.  Note 
that  no  false  point  detections  were  added  to  any  of  the  sets. 

The  resulting  edge  sets  are  shown  in  Figure  3-11.  Two  40%-reliable  sets 
were  created  and  three  20%-reliable  sets  were  created.  The  registration  re¬ 
sults  are  presented  in  Table  3-2.  For  each  experiment  the  best  2  transforma¬ 
tions  and  their  weights  are  reported.  The  second  column  shows  the  number  of 
points  chosen  by  the  random  selection  process.  The  last  column  lists  a  dif¬ 
ference  in  position  on  the  image  caused  by  the  difference  in  the  transforma¬ 
tion  found.  Except  for  the  case  where  no  corresponding  abstract  edges  were 
present,  the  transformations  obtained  were  very  consistent  with  the  "true" 
transformation  and  were  within  the  estimated  tolerances. 


3.3.2  Registration  of  A  Stereo  Pair 

The  second  set  of  registration  tests  used  the  stereo  pair  Dreux  12  and 
Dreux  13,  shown  in  Figures  1.2  and  1.3.  as  in  the  case  for  the  image  4621, 
maps  of  abstract  edges  of  the  two  images  were  created  using  a  Talos  digitizer. 
Registering  the  known  corresponding  abstract  edges  gave  the  "true"  transforma¬ 
tion  (0=19±1° , Ax=-4221±30, Ay=-1519±80) . 

A  set  of  real  edges  were  created  for  each  image  using  the  Talos  digitizer 
and  these  sets  were  registered  using  the  registration  program.  The  edges  sets 
are  shown  in  Figure  3.12  and  the  results  of  the  registration  are  shown  in 
Figure  3.13.  The  best  transformation  was  (19°, -4207, -1505)  which  is  well  with¬ 
in  the  variability  of  the  true  transformation.  For  an  image  point  of  (1000, 
1000)  the  difference  in  transformed  positions  would  be  20  stagels  or  .010  inches 
on  the  image. 
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(e)  20%  set. 
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(g)  20%  set. 


Figure  3.11  continued 
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Table  3.2 

Results  of  Robustness  Experiments 


Reliability 

Number  of  * 
Points  Selected 

Transformation*** 

(0,Ax,Ay) 

Weight 

Matched 

Image 

Edges 

Matched 

Map 

Edges 

Error  in 
Transformation 
(inches) + 

80% 

27 

(309°, 572,-1352) 

.496 

HI 

m 

0.018 

(309°, 598, -1378) 

.494 

K8M 

mm 

0.0 

60% 

19 

(309°, 598, -1378) 

.494 

PH 

m 

0.0 

(309°, 624,-1404) 

.492 

:  .  : 

0.019 

40% 

15 

(309°, 572, -1404) 

n/ii 

hU 

0.019 

(309°, 598, -1378) 

■a 

n/ii 

0.0 

40% 

14 

(308°, 598, -1378) 

.060 

— 

2/30 

0.0 

(308°, 624, -1352) 

.059 

mm 

2/30 

0.019 

20% 

8 

(309°, 572, -1404) 

.059 

2/4 

0.019 

(309°, 520, -1352) 

.057 

2/4 

■  ■ 

0.041 

(309°, 598, -1378) 

.055 

2/4 

2/30 

0.0 

20%** 

6 

(331°, 6032, -2392) 

.033 

mm 

_  ** 

(331°, 6084, -2340) 

.033 

■ 

_  ** 

20% 

13 

(309°, 598, -1378) 

.244 

7/10 

— 

9/30 

0.0 

(309°, 572, -1404) 

.237 

7/10 

9/30 

0.019 

*  Total  Number  of  Points  was  33. 

**  Neither  of  the  image  abstract  edges  corresponded  to  any  of  the  map  abstract  edges. 

***  Variability  on  transformation  parameters  were  estimated  as:  e=±l°,  Ax=±60, Ay=±40. 

*  For  a  point  at  (1000,1000)  on  the  image. 
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Figure  3.L3  Results  obtained  from  registration  software  when  run 
with  set  of  real  map  edges  from  DRE.UX  13  and  set  of 
real  image  edges  from  DREUX  13 
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3.3.3  Conclusions  on  2-D  Registration 

The  previous  sections  presented  details  on  only  some  of  the  total  number 
of  experiments  performed.  Although  all  of  these  experiments  had  the  same  en¬ 
couraging  results,  only  a  limited  amount  of  automatic  processing  was  done 
and  only  a  small  number  of  images  were  used.  Despite  these  limitations,  the 
demonstrated  robustness  of  the  registration  procedure  leads  us  to  believe 
that  the  automatic  feature  detection  procedures  discussed  in  the  later  sections 
of  Section  2  are  good  enough  to  supply  the  primitive  features  needed. 

Specific  conclusions  from  the  2-D  registration  experiments  are  as  follows. 

•  Registration  of  stereo  images  with  limited  relief  can  be  achieved  quite 
effectively  using  a  linear  transformation  consisting  of  only  rotation 
and  translation.  (Same  scale  has  been  assumed.) 

•  Abstract  edges,  formed  from  pairs  of  accurately  determined  image  points, 
provide  for  a  fast  and  accurate  registration  procedure. 

«  Use  of  real  edges  was  successful  for  registration  even  when  the  edge 

structure  barely  revealed  the  image  content.  However,  the  registration 
procedure  was  much  less  efficient  and  accurate  than  when  abstract  edges 
were  used. 

The  registration  results  are  sufficiently  good  so  that  no  modification 
appears  necessary  in  the  immediate  future:  effort  should  be  spent  on  verifica¬ 
tion,  region  analysis,  and  feature  detection.  However,  possible  improvements 
to  the  registration  procedure  are  as  follows. 

•  The  mathematical  details  relating  program  tolerances  to  the  amount  of 
local  relief  in  the  imagery  and  the  error  inherent  in  the  detectors 
should  be  worked  out  to  the  point  of  specifying  operating  procedures. 
(Tolerances  were  obtained  somewhat  arbitrarity  in  the  reported  experi¬ 
ments  . ) 
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•  Thresholds  used  in  clustering  should  be  computed  by  a  strict  a 

priori  rule  and  should  be  stringently  applied  so  that  fewer  clusters 
are  examined  and  less  computing  done.  Similarly,  the  clustering 
bin  size  should  be  regulated  according  to  the  theory  of  Section 
3.2.2. 

Very  little  has  been  said  about  the  "match  matrix"  optionally 
available  from  the  registration  program.  The  match  matrix  is  a  matrix 
whose  I,J-th  element  is  the  degree  of  match  between  image  edge  element 
I  and  map  edge  element  J  (i.e.  EMATCH(1 , J) ) .  Since  the  matrix  indicates 
which  edges  were  in  the  map  but  without  evidence  in  the  image  and  vise— 
versa,  it  is  directly  useable  for  change  detection  and  verification. 

Verification  of  features  in  the  map  but  without  edge  evidence  in  the 
image  is  reported  in  Section  5. 
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3.4  Full  RST  Transformation 

The  position  and  orientation  of  primitive  features  such  as  intersections, 
high  curvature  points,  or  lines  can  accurately  be  determined,  but  it  is  diffi¬ 
cult  to  determine  their  sizes,  such  as  length.  The  basic  LNK  registration 
technique  uses  the  position  and  orientation  of  the  primitive  features  to  find 
the  rotation  and  translation  necessary  to  register  two  images  or  to  register 
an  image  to  a  map. 

If  the  size  could  be  included,  then  it  would  be  possible  to  calculate 
the  four  parameter  transformation  of  rotation,  translation  and  scaling.  This 
section  presents  a  method  for  extending  the  procedure  to  account  for  scaling. 

In  order  to  accomplish  this,  abstract  vectors  or  edges  whose  size  can  accurately 
be  determined  are  introduced. 

To  achieve  scaling,  abstract  vectors  or  edges  can  be  formed  by  spanning 
pairs  of  point  structures.  For  example,  abstract  edges  can  be  formed  by  con¬ 
necting  pairs  of  high  curvature  points.  Abstract  vectors  could  be  formed  by 
using  an  intersection  point  as  the  vector  tail  and  a  high  curvature  point  as 
the  vector  head.  There  are  many  ways  of  forming  the  abstract  edges  or  vectors. 

The  registration  procedure  is  similar  to  the  basic  one.  Instead  of  the 
triples  (0,  Ax,  Ay),  there  are  four  parameters  (0,  Ax,  Ay,  As),  where  As  is 
the  scaling  parameter. 

The  three  registration  steps  are  now: 

1)  Primitive  point  features  (intersections,  high  curvature  points, 
etc)  are  automatically  extracted.  The  abstract  vectors  are 
created  by  pairing  the  primitive  point  features.  Not  all  pos¬ 
sible  pairs  need  be  formed  as  that  would  result  in  a  great  deal 
of  computation. 

2)  Assume  all  features  of  one  type  can  correspond  to  one  another. 

That  is,  a  vector  from  an  intersection  of  three  lines  to  an 
intet section  of  four  lines  in  the  first  image  can  correspond 
to  any  3- intersection  to  4-intersection  vector  extracted  from 
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the  map  (or  second  image).  For  each  possible  correspondence,  find 
the  4-parameter  transformation  (A3,  Ax,  Ay,  As)  that  maps  one  vector 
to  the  other.  Place  a  unit  of  weight  in  the  bin  in  the  four  dimen¬ 
sional  histogram  that  represents  the  (AO,  Ax,  Ay,  As)  found. 

3)  Locate  the  best  cluster  in  the  histogram.  The  (A3,  Ax,  Ay,  As)  of 
that  cluster  is  the  best  global  transformation  as  it  provided  the 
largest  number  of  local  correspondences. 

As  an  example  of  this  method,  suppose  an  image  and  map  are  represented  in 
terms  of  vectors  v.  connecting  intersection  points.  The  intersection  points  are 
of  four  types;  'L',  'T',  'X',  or  'Y'.  The  rules  for  pairing  points  to  form 
vectors  may  be  arbitrary;  for  example,  ' T'  points  are  joined  with  'L'  points. 

The  purpose  of  such  rules  is  to  control  the  combinatorics.  In  Figure  3.14,  five 
vectors  represent  the  map  and  four  vectors  represent  the  image. 

Given  any  map  vector  vj  all  possible  matching  image  vectors  are  con¬ 
sidered.  Each  possible  pairing  (v  .  ,v .)  results  in  an  RS&T  transformation 

J  ^ 

mapping  image  vector  v^  onto  map  vector  v_. .  The  transformation  is  specified 
by  a  quad  of  parameters  (AG,  Ax,  Ay,  As)  where  A3  is  the  angle  of  rotation. 

As  is  the  scale  factor,  and  Ax  and  Ay  are  the  x  and  y  translations  respectively. 
The  pair  (vj»vi>  discarded  without  producing  a  quad  if  the  tips  or  tails  of 
the  vectors  v  and  disagree  in  type.  In  the  example  of  Figure  3.14  there 
are  5x4»20  pairs  (v^v^)  initially  possible  and  of  these  only  10  agree  in  type 
of  tip  and  tail. 

The  mathematical  development  for  forming  a  quad  (AG,  Ax,  Ay,  As)  as  a 
function  of  and  is  given  in  Figure  3.13.  Table  3.3  shows  computer  out¬ 
put  for  the  example  shown  in  Figure  3.14.  There  are  10  quads  produced  and  a 
cluster  of  size  3  is  evident  in  the  neighborhood  of  the  best  registration 
transformation  (AG®5.10,  Ax“-75,  Ay»88,  As“0.5).  In  real  world  cases  there 
would  be  hundreds  of  quads  overall  and  a  few  dozen  in  a  cluster. 
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Type 


Code 


L  1 


T  2 

X  3 

Y  4 


Map  vectors 


(L, 170, 220) 
(T, 100, 100) 
(L, 200,100) 
(L, 260, 70) 
(T, 150, 125) 


-•  (X,  100, 200) 

-  (Y, 40, 150) 

-  (X, 220, 170) 

-  (X.40,70) 

-  (Y, 150, 50) 


Image  Vectors 


(L, 545, 400) 
(T, 260, 240) 
(T, 140, 380) 
(L, 420, 370) 


(X, 200, 120) 
(Y, 100,245) 
(Y, 300,380) 
(X, 360, 500) 


Figure  3.14.  a)  Example  set  of  map  vectors,  and 


b)  set  of  vectors  representing  an  image  to  be  registered 
to  the  map  by  RS&T  transformation. 
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Assuming  that  vector  corresponds  to  vector  Vj  transformation 
parameters  (A0, Ax, Ay, As)  are  gotten  as 


A0  -  0,  -  0. 
J  i 


As  »  length  of  /length  of  v.^ 

Ax  -  As  Ay  sinA0  —  As  Ax  cosAG  +  Cx 


Ay  55  -As  Ax  sinA0  -  As  Ay  cosAG  +  Cy 


The  resulting  registration  transformation  in  homogeneous  coordinates 
is 


[ufv,l]  -  [x,y ,1] 


As  cosA0  As  sinAG  0 

-As  sinAG  As  cosAG  0 

Ax  Ay  1 


where  (x,y)  is  an  image  point  and  (u,v)  is  the  corresponding  map  point. 


Figure  3.15.  Mathematical  derivation  of  RS&T  transformation  parameters 
from  a  pair  of  vectors  v.  and  V.  assumed  to  be  correspond¬ 
ing.  2 
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3.4.1 


Example  with  Scale 


The  following  experiment  demonstrates  the  utility  of  the  proposed  technique 
for  regions  where  cultural  activity  creates  features  such  as  straight  edges 
or  networks  of  lineals.  The  results  reported  are  typical  of  many  similar  ex¬ 
periments  . 

Point  features  were  identified  by  eye  on  an  aerial  image  from  the  mid- 
western  U.S.A.  using  two  different  measuring  devices  and  two  different  orien¬ 
tations.  Figure  1.1  shows  sample  imagery  while  Table  3.1  contains  the  coor¬ 
dinates  of  the  selected  feature  points.  Points  are  labeled  according  to  the 
type  of  road  intersection  -  'L' ,  'T',  'X' ,  or  'Y'  -  which  they  represent  or 
are  labeled  'A'  indicating  an  arbitrary  point  on  the  road. 

50  vectors  were  chosen  to  model  the  map  of  Figure  1.1  while  64  vectors 
represented  the  image.  Tables  3.4  and  3.5  show  some  of  the  resulting 
quads  formed  by  matching  vectors  from  the  map  with  vectors  from  the 
image.  Note  that  stage  coordinates  have  been  divided  by  10  for  format  con¬ 
venience.  Table  3.4  shows  10  vectors  in  stage  coordinates  which  have  the  same 
type  of  tip  and  tail  as  the  vector  (4365,2747)  -  (2498,3641)  in  the  photo  model. 
Only  the  first  of  these  matches  is  correct  and  hence  only  the  first  quad  (5.39, 
-1020,4040.  5.01)  contributes  to  the  ultimate  cluster.  Of  the  50  x  64  =  3200 
pairs  (Vj.v^)  possible  only  790  nroduce  quads  in  cluster  space  after  the  check 
on  tip  and  tail  type.  Table  3.5  shows  that  32  of  the  790  quads  form  a  cluster 
near  the  parameter  set  (A0»5.38,  Ax*«-1000.  ,  Ay*4000. ,  As=5.0).  Using  Table  3.5 
che  reader  can  verify  that  30  of  these  32  quads  represent  correct  vector  matches. 
For  instance,  the  first  quad  represents  the  matching  of  vectors  from  point  /(2 
to  point  //l  in  the  two  different  coordinate  systems.  The  asterisks  mark  the 
two  incorrect  matches  which  in  fact  are  outliers  in  the  set  of  32  quads.  A 
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Table  3.4. 


First  10  quads  in  set  of  790  quads  produced  by 
comparing  50  map  vectors  with  64  image  vectors. 
(Stage  coordinates  of  Table  4-4  divided  by  10 
for  format  convenience). 


MAP  VECTOR  IMAGE  VECTOR  TRANSFORMATION 


VEC 

TAIL 

HEAD 

VEC 

TAIL 

HEA0 

a 

TTPCX, Y» 

TYP(X,Y) 

I 

YYP(X,Y) 

TYP(X, Y) 

THETA 

SCALE 

0ELX 

DELT 

i 

3(4345,2747) 

4(2498,3441) 

51 

3( 

872,  480) 

4( 

500, 

500) 

.539*01 

.501+01 

-.102+04 

.404+04 

i 

3(4345,2747) 

4(2498,3641) 

52 

3< 

391,  720) 

4( 

500, 

500) 

.381+01 

.843+01 

.322+04 

.956+04 

i 

3(4345,2747) 

4(2498,3641) 

53 

3  ( 

331,  840) 

4  ( 

500, 

500) 

.380+01 

.545+01 

.297+04 

.747+04 

i 

3(4345,2747) 

4(2498,3441) 

54 

3< 

281,  951) 

4  ( 

500, 

500) 

.381+01 

.413+01 

.283+04 

.654+04 

i 

3(4345,2747) 

4(2498,3641) 

55 

3( 

350,  994) 

41 

500, 

500) 

.397+01 

.401+01 

.237+04 

.647+04 

i 

3(4345,2747) 

4(2498,3641) 

56 

3( 

656,1144) 

4( 

500, 

500) 

.450+01 

.312+01 

.129+04 

.549+04 

i 

3(4345,2747) 

4(2498,3641 ) 

61 

3(1001,1310) 

4  ( 

500, 

500) 

.482+01 

.217+01 

.130+04 

.461+04 

i 

3(4345,2747) 

4(2498,3641 ) 

62 

3( 

872,  680) 

4(1112, 

805) 

.221+01 

.765+01 

.125+05 

.536+03 

i 

3(4345,2747) 

4(2498,3641 ) 

67 

3< 

872,  630) 

4  ( 

171, 

1360) 

.324+00 

.212+01 

.307+04 

.793+03 

i 

3(4345,2747) 

4(2498,3641 ) 

68 

3( 

391,  720) 

4(1112, 

805) 

.258+01 

.285+01 

.640+04 

.389+04 

32  quads  contained  in  the  bin  0e[5.O,5.7],  Se[4. 8,5.2], 
Ax  e  [-1100,-900],  and  Ay  e [3000,5000]. 


MAP  VECTOR 

VEC  TAIL  HEAD 

it  TYP(X.Y)  TYP (X, Y) 

1  3(4365,2747)  4(2498,3641) 

2  3(3044,4744)  4(2498,3641) 

3  3(3342,5357)  4(2498,3641) 

4  3(3611,5908)  4(2498,3641 
4  3(3611,5908)  4(2498,3641) 
6  3(3975,5750)  4(2498,3641) 

6  3(3975,5750)  4(2498,3641) 

7  3(5516,5073)  4(2498,3641) 

8  4(2498,3641)  5(7130,3514) 

9  1(7430,4091)  4(2498,3641) 

10  1(7723,4700)  4(2498,3641) 

11  1(6942,3613)  4(2498,3641) 

12  3(7233,4181)  4(2498,3641) 

19  3(4365,2747)  4(5583,2191) 

20  3(4365,2747)  5(7130,3514) 

21  1(7430,4091)  3(4365,2747) 

22  1(7723,4700)  3(4365,2747) 

24  1(6942,3613)  3(4365,2747) 

25  3(4365,2747)  4(4861,7575) 

29  3(3044,4744)  4(5583,2191) 

30  3(3342,5357)  4(5583,2191) 

31  3(3611,5908)  4(5583,2191) 
33  3(3975,5750)  4(5583,2191) 

35  4(5583,2191)  5(7130,3514) 

36  1(7430,4091)  4(5583,2191) 

37  1(7723,4700)  4(5583,2191) 

38  1(6942,3613)  4(5583,2191) 

39  3(7233,4181)  4(5583,2191) 

46  3(3044,4744)  5(7130,3514) 

47  1(7430,4091)  3(3044,4744) 

48  1(7723,4700)  3(3044,4744) 
50  1(6942,3613)  3(3044,4744) 


IMAGE  VECTOR 

VEC  TAIL  HEAD 

it  TYP(X.Y)  TYP  (X,Y) 

51  3 (  872,  680)  4(  500,  500) 

52  3(  391,  720)  4(  500,  500) 

53  3(  331,  840)  4(  500,  500) 

54  3(  281,  951)  4(  500,  500) 

55  3 (  350,  994)  4(  500,  500) 

54  3(  281,  951)  4(  500,  500) 

55  3(  350,  994)  4(  500,  500) 

56  3(  656,1144)  4(  500,  500) 

57  4(  500,  500)  5(1096,1210) 

58  1(1036,1330)  4(  500,  500) 

59  1(  986,1460)  4(  500,  500) 

60  1(1056,1200)  4 (  500,  500) 

61  3(1001,1310)  4(  500,  500) 

62  3(  872,  680)  4(1112,  805) 

63  3(  872,  680)  5(1096,1210) 

64  1(1036,1330)  3(  872,  680) 

65  1(  986,1460)  3(  872,  680) 

66  1(1056,1200)  3(  872,  680) 

67  3(  872,  680)  4(  171,1360) 

68  3(  391,  720)  4(1112,  805) 

69  3 (  331,  840)  4(1112,  805) 

70  3(  281,  951)  4(1112,  805) 

71  3 (  350,  994)  4(1112,  805) 

73  4(1112,  805)  5(1096,1210) 

74  1(1036,1330)  4(1112,  805) 

75  1(  986,1460)  4(1112,  805) 

76  1(1056,1200)  4(1112,  805) 

77  3(1001,1310)  4(1112,  805) 

78  3(  391,  720)  5(1096,1210) 

79  1(1036,1330)  3(  391,  720) 

80  1(  986,1460)  3(  391,  720) 

81  1(1056,1200)  3 (  391,  720) 


TRANSFORMATION 


THETA  SCALE  DELX  DELY 

.539+01  .501+01-. 102+04  .404+04 
.536+01  .501+01-. 101+04  .412+04 
.536+01  .504+01-. 103+04  .411+04 
.537+01  .504+01-. 104+04  .408+04 
.553+01  .489+01-. 959+03  .352+04 
.522+01  .514+01-. 994+03  .464+04 
.538+01  .499+01-. 100+04  .406+04 
.539+01  .504+01-. 105+04  .401+04 
.538+01  . 500+01- . 101+04  .404+04 
.538+01  .501+01-. 102+04  .407+04 
.538+01  .495+01-. 982+03  .405+04 
.538+01  .497+01-. 992+03  .406+04 
.538+01  .500+01-. 102+04  .406+04 
.537+01  .495+01-. 941+03  . 408+04 
.538+01  .499+01-. 994+03  .405+04 
.537+01  .499+01-. 987+03  .410+04 
.538+01  .493+01-. 933+03  .402+04 
.538+01  .493+01-. 923+03  .406+04 
.538+01  .497+01-. 972+03  .406+04 
.538+01  .496+01-. 962+03  .407+04 
.537+01  .496+01-. 956+03  .410+04 
.537+01  .499+01-. 992+03  .410+04 
.538+01  .497+01-. 986+03  .406+04 
.538+01  .502+01-. 105+04  .407+04 
.537+01  .500+01-. 991+03  .414+04 
.539+01  .494+01-. 959+03  .400+04 
.538+01  .493+01-. 927+03  .404+04 
.537+01  .500+01-. 101+04  .410+04 
.538+01  .497+01-. 967+03  .404+04 
.538+01  .499+01-. 991+03  .406+04 
.538+01  .493+01-. 935+03  .406+04 
.538+01  .495+01-. 955+03  .408+04 


4-D  clustering  routine  would  need  to  be  implemented  to  find  the  significant 
clusters. 


3.5  Extending  LNK  Registration  to  3  Dimensions 

So  far  the  LNK  registration  procedures  have  ignored  the  3D  aspects  of 
the  objects.  This  section  presents  a  method  to  extend  the  2D  registration 
to  3D  under  certain  constraints.  The  2D  procedure  found  RS&T  transforma¬ 
tions  mapping  map  structure  onto  image  structure.  Experiments  showed  that 
the  RS&T  assumption  was  feasible  in  cases  where  variation  in  the  3rd  dimen¬ 
sion  was  relatively  small.  Good  approximate  RS&T  registration  transforma¬ 
tions  were  obtained  automatically  for  photo/map  pairs  even  when  there  was 
some  relief  in  the  terrain.  There  are  many  cases  where  RS&T  transformations 
are  inadequate,  such  as  in  low  altitude  aerial  imaging  and  in  acquisition  cf 
solid  objects  by  a  robot  vision  system.  In  these  cases  projective  transforma¬ 
tions  must  be  used. 

In  the  general  case  6  parameters  are  necessary  to  specify  imaging  in  a 
3D  world  [Duda  and  Hart  1973].  In  this  section,  we  consider  a  constrained 
imaging  system  with  only  3  free  parameters  as  shown  in  Figure  3.16.  A  front 
image  plane  is  used  with  reference  system  origin  (x=0,y=»0,z=0)  at  the  image 
center.  The  camera  has  known  focal  length  f  and  looks  vertically  down  at  a 
scene  with  base  distance  y0  from  the  image  plane.  There  are  only  3  unknown 
parameters  to  discover;  the  angle  0  at  which  the  object  lies  on  the  base 
plane  and  the  amount  of  translation  (x0, z0)  of  the  object  origin  from  the 
point  where  the  camera  axis  pierces  the  plane. 

There  is  some  justification  for  this  assumption  in  the  aerial  imaging 
case.  First,  f  is  usually  known.  Second,  it  is  possible  to  get  a  good  ap¬ 
proximation  for  altitude  y0  and  to  achieve  a  nearly  vertical  camera  axis. 

These  approximations  would  perhaps  be  good  enough  to  correctly  detect  an 
approximate  3  parameter  view  which  could  be  used  as  an  initial  approximation 
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Figure  3.16.  Three  dimensional  object  viewed  by  camera  with 

known  attitude.  Knowns  are  f  and  YQ  and  a  model 
of  the  object.  Unknowns  are  the  object  orienta¬ 
tion  parameters  0,  X„,  and  Z„. 


to  a  full  6  parameter  view. 

We  therefore  proceed  as  follows.  A  3D  terrain  model  is  specified  in 
local  coordinates  as  in  Figure  3.17.  The  model  would  contain  the  location 
and  description  of  all  significant  features  such  as  edges,  corners,  inter¬ 
sections,  water  bodies,  etc. 

The  acquisition  problem  is  defined  as  discovering  (computing)  the  orien¬ 
tation  parameters  (9,x0,z0)  from  the  image  structure  and  the  known  camera 
parameters  f  and  yQ.  A  few  definitions  are  appropriate  before  proceeding. 

camera  parameters  -  parameters  that  fix  the  imaging  system  over  the 
base  plane,  i.e.  f  and  y0,  and  define  the  global 
coordinate  system. 

orientation  parameters  -  parameters  that  fix  the  object  (or  object  model) 

in  the  global  coordinate  system  which  are  9,  x0, 
z0. 

viewing  parameters  -  the  combined  camera  and  orientation  parameters  f,yQ, 

QjXq.Zq  which  allow  a  specific  image  to  be  created 
(from  the  terrain  model). 

Here  we  assume  that  we  know  the  attitude  of  the  camera  f,y0  and  the  ori¬ 
entation  9,x0,zo  of  the  object.  We  develop  computational  formulas  for  image 

point  (x_,yT,zT)  corresponding  to  point  (x  ,y  ,z  )  in  the  map. 

Ill  rn  mm 

Let  (x,y,z)  be  the  global  coordinates  of  point  (xm,ym,zm)  under  map 
orientation  (9,x0,z0).  Then  we  have 

(1)  x  ■  x_  cos  0  -  z  sin  6  +  x„ 

in  n  o 

(3D  Map  to 

(2)  y  -  yQ  -  ym 

ju  Global) 

(3)  z  *  Xjn  sin  9  +  zm  cos  9  +  z0 

The  global  coordinates  (x,y,z)  then  produce  image  coordinates  (x^y^z^) 
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Here  we  assume  that  a  given  vector  (A^,  Aym,  Azm)  -  (Bj^,  Bym,  Bz^,)  in 
the  map  corresponds  to  a  given  vector  (Ax’,  Az')  -  (Bx',  Bz’)  in  the  image. 

We  develop  computational  formulas  for  determining  map  orientation  parameters 
(o,x0,y0)  from  this  correspondence. 

Rearranging  the  imaging  equations  (4)  and  (6)  from  above  we  have  the 
following. 

(7)  xI(f  ~  ym  +  yo>  *  Xm  cos  9  ~  zm  sin  9  +  xo 

£ 

(8)  zx(f  -  ym  +  y0)  -  Xm  sin  6  +  zm  cos  9  +  zo 

f 

Since  (7)  must  hold  for  both  points  A  and  B,  we  get  two  equations  from 
which  xQ  can  be  eliminated,  leaving  only  0  unknown. 

(9)  Axt(f  -  Aym  +  yQ)  -  Bxx(f  -  Bym  4-  yQ)  =  (Ax^,  -  Bxm)cos  6  + 

f  f 

(Bz_  -  Az_)sin  9 
ra  m 

Equation  (9)  is  of  the  form 

c  *  d  cos  9  +  e  sin  0 

where  we  make  the  substitutions  w  -  sin  0  and  ■  COS  9. 

Thus  a  standard  quadratic  equation  in  w 

(10)  (e2  +  d2)w2  -  (2ce)w  +  (c2  -  d2)  -  0 

is  obtained  where  the  coefficients  are  obtained  as  follows: 


1 19 


c  -  (AXj  (f  -  Aym  +  yQ)  -  BxI(f  -  Bym  +  yQ))/ 

d  “  Axm  -  Bxm 
e  -  Bzm  -  Azm- 

Solving  the  quadratic  yields 

(11)  w  »  ce  +  d Ve^  -  +  d^ 


(12)  0  =  sin-1  w. 

Knowing  0,  (7)  and  (8)  can  be  used  to  solve  for  xQ  and  yQ  using  either  A  or 
B  point  coordinates.  For  example, 

xo  *  AxI(£  -  Aym  +  y0)/f_Axm  cos  6  +  Azm  sin  9- 

Since  there  is  mathematical  ambiguity  in  0  from  (11)  two  parameter  sets 

^wl*x01’y01)  and  ^w2,x02,y02^  Can  resu^t*  is  easy  t0  check  for  correct 

parameters  using  (4)  and  (6)  and  the  2  known  pairs  of  corresponding  points 

(AI’V  and  (BI,Bm)  * 

There  are  two  significant  cases  to  note  where  the  computation  breaks 
down.  First  of  all,  the  discriminant  of  the  quadratic  can  be  negative  and 
hence  no  0  can  be  obtained.  This  will  happen  whenever  the  map  edge  cannot 
possibly  be  imaged  onto  the  image  edge.  Few  pairings  are  actually  possible 
due  to  the  fixed  scale  imposed  by  f  and  yQ.  Secondly,  whenever  the  map  edge 

is  vertical  both  d  and  e  above  are  zero  and  equations  (11)  and  (12)  cannot 

produce  9.  Physically  we  can  rotate  the  object  in  the  map  freely  about  that 
vertical  edge  without  altering  its  image  and  thus  we  should  not  expect  to  get 
0  mathematically  either. 

Figure  3.18  shows  a  planar  section  containing  the  camera  axis,  the  ver¬ 
tical  map  vector,  and  hence  the  image  vector  as  well.  Clearly,  free  rotation 
of  the  object  about  the  axis  AB  will  not  change  this  picture.  It  is  also 
clear,  however,  that  locational  parameters  x  and  z  are  completely  specified 
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Figure  3.18.  Planar  section  of  imaging  environment  containing  both 
the  camera  axis  and  a  verticle  model  vector  AB. 


i 


118 


by  A'B'  and  AB  provided  Chat  they  correctly  match  via  the  viewing  transforma¬ 
tion.  From  the  imaging  equations  (4)  and  (6)  applied  to  both  points  A  and  B 
we  get 

AXj(f+y0)  /  f  ■  x  -  Bx^f+y)  /  f  or 

(23)  AxI(f+yo)  -  BXjCf+y) 

and  similarly  for  the  z  coordinate  relations 

(24)  AzI(f+yQ)  =  BZj(f-t-y)  . 

Conditions  (23)  and  (24)  must  hold  if  A'B'  is  to  possibly  match  with  AB. 

We  are  already  in  trouble  here  if  only  real  edge  segments  are  available 
because  (23)  and  (24)  are  scaling  equations.  On  the  other  hand,  if  abstract 
vertical  map  edges  which  have  accurate  tips  and  tails  are  being  used,  it  would 
be  easy  to  consider  only  nonvertical  vectors  as  before.  All  that  need  be  done 
is  to  construct  nonvertical  vectors  by  mixing  the  tip  and  tail  points  of 
several  vectors.  For  this  reason,  special  treatment  of  vertical  vectors  can 
be  ignored  (justifiably)  in  the  computer  programs. 

As  an  example,  suppose  the  map  contained  the  object  shown  in  Figure  3.17. 
Suppose  the  image  was  formed  by  viewing  the  object  with  aspect  parameters 
(f"l.yo*10)  and  orientation  parameters  (0-30°, xQ— 2,zq-3) .  The  resulting 
image  is  shown  in  Figure  3.19,  along  with  the  coordinates  of  the  points.  Let 
the  object  in  the  map  and  image  be  represented  by  the  vectors  listed  in  Table 
3.6.  Matching  the  10x10  pairs  of  vectors  (v^v^)  yields  only  12  feasible  para¬ 
meter  sets,  10  of  which  form  a  cluster  about  the  correct  parameters  a  *  (0»O.52 
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Figure  3.19.  Object  model  defined  by  set  of  vertices  and  edges. 
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4.  Region  Identification 

The  interpretation  of  images  frequently  necessitates  the  extraction 
and  identification  of  significant  regions.  This  requires  finding  the  bound¬ 
aries  of  the  regions  and  assigning  labels  which  identify  the  regions'  con¬ 
tents.  The  segmentation  of  images  into  regions  was  briefly  presented  in 
Section  2.8.  The  identification  of  regions  can  be  thought  of  as  a  partitioning 
process . 

Given  a  segmentation,  the  first  step  of  region  identification  is  to  find 
a  set  of  features  that  would  be  used  to  provide  an  initial  set  of  feasible 
labels  for  the  regions.  This  discrimination  could  be  done  using  statistical 
or  structural  pattern  recognition  procedures.  Since  a  region  could  be  multi¬ 
ply  classified,  the  second  part  of  region  identification  involves  disambiguat¬ 
ing  the  labels  using  either  inter-region  relationships  and/or  structural  infor¬ 
mation  from  a  map. 

In  Section  4.1,  we  present  statistical  methods  which  use  the  feature 
vector  to  provide  an  initial  set  of  labels  for  the  regions.  Experimental 
studies  comparing  several  of  the  methods  are  described. 

In  Section  4.2,  we  present  a  brief  overview  of  structural  methods  for  re¬ 
ducing  the  ambiguity  of  the  region  labelling.  A  graph  theoretical  method  of 
region  matching  between  an  image  and  a  map  is  described.  The  technique  uses 
network  flow  analysis  to  perform  the  disambiguation.  A  second  method  which 
uses  an  interative  procedure  based  on  similarity  measures  between  neighboring 
regions  is  sketched. 
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4.1  Region  Label  Weights 


This  section  provides  an  overview  of  the  problem  of  assigning  a  set  of 
labels  and  associated  likelihoods  or  weights  to  these  labels  given  a  feature 
vector  describing  the  region.  The  labels  will  be  various  classes  such  as 
forest,  field,  etc.  into  which  the  region  can  be  classified.  Ideally  the  weights 
should  be  the  conditional  probability  of  the  region  feature  vector  given  each 
of  the  classes.  Unfortunately  the  estimation  of  these  quantities  is,  in  general, 
very  difficult  without  a  large  number  of  samples.  The  weights  described  in  this 
section  can  be  obtained  by  using  training  samples  to  form  a  model  for  each 
class  and  taking  as  weights  some  measure  of  fit  of  the  region  feature  vector  to 
the  model.  The  models  discussed  here  fall  into  two  categories.  First  we 
describe  models,  called  single  models,  which  are  defined  using  only  training 
samples  from  the  class  being  modelled.  Second  we  describe  models,  called 
multi  models,  which  are  extracted  from  classification  procedures  using  training 
samples  from  two  or  more  classes  simultaneously. 

A  class  model  defined  solely  in  terms  of  the  samples  from  that  class  has 
several  advantages  over  a  class  model  based  on  training  samples  from  several 
classes.  First,  the  addition  of  new  classes  can  be  easily  accomodated  by  using 
samples  from  these  new  classes  to  form  the  new  models.  The  models  of  the 
previously  existing  classes  would  require  no  change,  thus  providing  a  significant 
reduction  in  computation  and  data  storage  requirements  over  multi  models.  A 
second  advantage  of  single  models  over  multi  models  is  the  improved  quality  of 
class  fitting.  Multi  models,  since  they  are  based  on  classification  methods, 
attempt  to  use  variables  which  are  best  suited  to  class  discrimination,  and 
use  the  same  variables  for  all  classes  being  modeled.  These  multi  methods  may 
require  the  use  of  variables  irrelevant  to  certain  classes  thus  distorting  the 
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estimation  of  separating  surfaces.  Single  models,  on  the  other  hand,  develop 
class  models  using  only  those  variables  relevant  to  the  description  of  the 
class  involved.  A  third  advantage  of  single  models  over  multi  models  Is 
their  behavior  in  the  presence  of  test  samples  from  new  classes  for  which  no 
training  samples  are  available.  In  single  models  the  sample  from  the  new 
class  might  not  fit  any  model  well  in  which  case  it  could  be  easily  isolated 
for  further  examination.  Multi  methods  use  the  partition  of  feature  space 
given  by  the  classification  procedure  and  some  measure  of  distance  of  a  sam¬ 
ple  from  each  set  in  the  partition.  In  this  case  the  partition  is  not  adequate 
to  design  separation  measures  which  detect  the  presence  of  samples  from  a  new 
class  since  all  samples  fall  within  some  set  of  the  partition. 

Multi methods  can  be  advantageous  in  situations  where  variables  good  for 
class  separation  are  present  but  variables  used  in  the  single  models  contain 
little  information  useful  for  discrimination.  Unfortunately,  the  characteri¬ 
zation  of  data  for  the  purpose  of  determining  the  relevance  of  single  versus 
multi  models  is  not  all  understood.  In  Section  4.1.1  to  4.1.5  we  examine 
specific  models  illustrating  by  example  certain  pitfalls  with  those  methods. 

Section  4.1.1  describes  the  disjoint  principal  components  model  which  is  a 
single  model.  Section  4.1.2  details  the  pooled  covariance  discrimination 
model  which  is  a  multi  model.  In  Sections  4.1.3  thru  4.1.5,  experiments 
using  these  models  are  described. 
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4.1.1  Disjoint  Principal  Components  Model 


The  disjoint  principal  components  model  is  formed  by  fitting  separate 
principal  components  to  each  class  and  using  the  distance  from  a  test  sample 
to  the  subspace  of  feature  space  given  by  the  principal  components  models  as 
the  measure  of  membership  in  the  class.  Thus,  smaller  weights  indicate  greater 
likelihood  of  class  membership.  In  this  section  we  review  the  theory  of 
principal  components  and  describe  the  disjoint  principal  components  methods 
in  detail,  indicate  some  of  its  limitations  and  provide  suggestions  for  im¬ 
provement  of  the  procedure. 

The  goal  of  principal  components  [Kshirsagar  1972]  is  to  give  a  simple 
model  for  a  continuous  multivariate  distribution  which  maximizes  the  explained 
variance  of  the  distribution.  This  goal  has  been  stated  in  a  very  general  form 
since  there  are  at  least  eight  distinct  natural  optimization  questions  whose 
solutions  are  the  principal  components.  Before  describing  some  of  these  char¬ 
acterizations,  we  define  principal  components  and  give  some  basic  properties. 

Let  x=(x^...Xp)  be  a  sample  from  a  p-dimensional  continuous  distribution  with 
mean  0  and  covariance  matrix  £=(0^)  where  £  is  a  real  positive  semidefinite 
matrix  with  eigenvalues  6j,  .  .  .  ,  6p  where  6 ^^2—  *  •  •  ^>^0.  There  exists 
a  pxp  orthogonal  matrix  T  whose  columns  Yp***»Yp  are  the  eigenvectors  of  £  such 
that  £=rAT'  or  r'Er=A  and  A=diag  (6^,...,6  ),  is  the  matrix  with  entries 
6p...,dp  along  the  main  diagonal  and  zero  elsewhere  and  denotes  the  trans¬ 
pose  of  a  matrix.  For  any  xeRP,  we  define  the  principal  components  vi>---»vp 
of  x  to  be  the  components  of  the  vector  v=T'  x.  Intuitively  we  have  merely 
performed  a  change  of  basis  to  the  basis  in  which  the  eigenvectors  of  £  are 
the  new  basis  vectors  given  in  terms  of  decreasing  eigenvalues.  Thus  cor¬ 
responds  to  the  eigenvalue  6]_,  v2  corresponds  to  62,  etc.  The  principal  com¬ 
ponents  are  uncorrelated  and  their  variances  are  62,...,6p  respectively. 
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We  now  describe  several  optimal  properties  of  principal  components. 


First,  suppose  we  wish  to  find  q  (q<p)  linear  combinations  of  x^,...,x 
which  account  for  as  much  of  the  variation  of  as  possible.  Thus 

we  seek  a  q  by  p  matrix  H  whose  rows  h^,...,h  are  orthonormal  such  that  the 
sum  of  the  variances  of  h^'x, . , . .h^'x  is  maximized.  It  can  be  shown  that 
h^v^  for  i=l,...,q.  Thus  the  first  q  principal  components  are  the  linear 
combinations  of  x^,...,Xp  explaining  the  maximum  amount  of  the  variance  of 
x^,...,x  .  We  give  one  other  optimal  property  of  principal  components  here. 
Suppose  that  instead  of  attempting  to  explain  as  much  of  the  variance  of  x  as 
possible  we  attempt  to  approximate  the  covariance  matrix  E  by  a  matrix  B  of 
the  same  order  p  and  rank  q  (<p)  .  We  wish  to  find  such  a  B  minimizing 

Norm(E-B)  =  E^O^-b^  )  ) 

where  B=(bij).  To  state  the  optimality  principle  here,  we  first  recall  that 
E  can  be  written  as 


E  -  <51Y1Y1'  +  <52Y2Y2’  +  ...  +  VpV’ 

according  to  the  spectral  decomposition  theorem.  It  can  be  shown  that  the 
optimal  matrix  B  is  given  by 


B  =  <5iYiYi'  +  .  .  .  +  6qYqYq'. 

This  matrix  has  eigenvectors  Y^,  ...,  Yq  and  eigenvalues  6^ . 6^  so  the 

principal  components  computed  with  this  approximate  matrix  are  just  the  first 
q  principal  components  computed  from  E. 

Various  hypotheses  can  be  tested  to  determine  the  number,  k-1,  of  princi¬ 
pal  components  to  select.  These  tests  assume  a  multivariate  normal  distribu¬ 
tion  for  the  random  vector  x.  Two  hypotheses  which  can  be  used  are 


1) 

.1  ,  6. 

is 

zero. 

i=p-k+l  1 

P 

,  P 

2) 

E  6 

/  ? 

is  zero. 

i=p-k+l  1 

l  1 

Hypothesis  (1)  states  that  the  sum  of  the  variances  of  the  last  k  principal 
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components  is  small  while  hypothesis  (2)  states  that  the  ratio  of  the  sum  ol 
the  last  k  variances  to  the  sum  of  all  the  variances  is  zero.  Both  hypotheses 
have  intuitive  appeal  as  measures  of  the  insignificance  of  the  last  k  princi¬ 
pal  components.  Details  of  these  tests  are  given  in  [Kshirsagar  1972]. 

The  principal  component  approach  to  distribution  representation  can  be 
adapted  to  the  problems  of  classification  and  ranking  of  class  labels  [Wold 
1976  ].  We  assume  k  classes  Cp...,^  are  present.  Each  class  Cp  i=l,...,k 
is  represented  by  a  collection  of  samples  where  a  sample  is  a  point  in  Rn. 

First  all  variables  in  each  class  are  modified  to  have  zero  mean  and  unit  vari¬ 
ance.  This  is  done  to  eliminate  the  effect  of  magnitude  difference  between 
variables  due  to  inappropriate  choices  of  scales  for  the  various  variables. 


For  each  class,  Cp  a  principal  component  model  is  formed  using  the  normalized 
variables.  The  number  of  principal  components  may  be  manually  or  automatically 
selected  using  any  of  a  range  of  tests  [Kshirsagar  1972].  Having  formed  these 


models  a  sample  can  be  tested  for  its  fit  to  each  of  the  models.  Let  m^  and 
s^j  denote  the  unnormalized  mean  and  variance  of  the  jth  variable  of  the  ith 
class.  Let  x  =  (x^,...,xn)  be  a  sample  to  be  fitted.  Define  a  class  i  nor¬ 


malized  vector  x'  =  (x,  ' , . . .  ,x_  ’)  of  x  by  x.'  =  -j~mi..i.  Let  x"  =  (x.  "  , . . . , 

J  sij  1 

x  ' ')  denote  the  projection  of  x'  on  the  space  spanned  by  the  eigenvectors  cor¬ 


responding  to  the  selected  principal  components  for  this  class.  Finally  we 


define  the  fit  of  x  to  class  i  to  be  the  Euclidean  distance  between  the  vec¬ 


tor  x  and  x' * .  Thus  we  have  a  measure  of  the  distance  of  a  sample  from  each 
class.  In  this  model  a  class  is  represented  by  a  linear  subspace  of  Rn.  The 
advantages  attributed  to  single  models  in  section  4.1  apply  to  the  disjoint 
principal  components  model.  The  selection  of  a  linear  subspace  to  represent 
a  class  can  be  lead  to  problems  in  classifying  a  sample  if  the  sample  lies 
close  to  a  representing  subspace  but  far  from  the  samples  used  to  select  this 
subspace.  Before  suggesting  a  modification  of  the  algorithm  to  overcome  these 
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difficulties,  if  they  arise,  we  remark  that  the  ranking  of  classes  for  a  sam¬ 
ple  is  defined  by  the  distance  from  a  class  in  the  obvious  way:  class  is 
more  likely  than  class  if  the  normalized  sample  is  closer  to  the  model  for 
than  it  is  to  the  model  for  Cj . 

A  refinement  of  the  principal  components  models  may  be  obtained  by 
modification  of  the  notion  of  the  distance  from  a  sample  to  a  class.  In  the 
subspace  chosen  by  the  disjoint  principal  components  method  to  model  a  class 
we  may  perform  a  parametric  or  non-parametric  density  estimation  for  the  pro¬ 
jections  of  the  samples  of  the  class  onto  the  subspace.  If  the  number  of 
principal  components  is  small,  this  estimate  may  be  quite  reliable.  The  amende! 
notion  of  the  distance  from  a  sample  to  a  class  should  be  small  if  the  sample 
is  near  a  point  in  the  subspace  which  has  a  high  probability  according  to  the 
density  estimate.  If  we  normalize  the  distance  of  a  sample  to  each  class  by 
dividing  the  ordinary  principal  components  distance  to  a  class  by  the  sum  of 
the  distances  to  all  the  classes,  and  we  denote  this  normalized  distance  by  d, 

then  we  can  define  the  new  distance  of  a  class  by  where  p  is  the  estimated 

P 

probability  of  the  projection  of  the  sample.  If  we  wish  to  use  inverse  dis¬ 
tances,  so  that  smaller  values  represent  better  fits,  we  may  use  i+p.  In  the 
case  of  equal  prior  probabilities  for  each  class  and  all  principal  components 
being  used  this  is  just  Bayes  rule. 
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4.1.2  Pooled  Covar lance  Discrimination 


The  posterior  probabilities  of  a  samp  1  e  be  loop,  i  up.  in  each  >i  several 
classes  can  be  used  to  rank  the  likelihoods  of  membership  in  each  of  the 
classes.  If  each  class  is  assumed  to  have  a  multivariate  normal  distribu¬ 
tion  and  all  classes  have  a  common  covariance  matrix,  then  the  estimation  of 
the  posterior  probabilities  is  considerably  simplified.  Software  for  the 
computation  of  these  probabilities  is  available  in  the  BMDP  package  [Dixon 
1979].  We  now  describe  the  computation  of  these  quantities.  For  further 
details,  see  [Dixon  1979],  Let  p  denote  the  number  of  variables,  p  the  num¬ 
ber  of  classes  used  for  design,  t  the  total  number  of  classes  and  x. .  the 

i.l  r 

value  of  variable  r  of  sample  j  of  class  i.  Furthermore  let  n  be  the  total 


number  of  samples  and  let  n^  be  the  number  of  samples  in  class  i.  In  the 
experimental  work,  each  class  was  equally  divided  into  two  classes,  one  for 


First  compute  the  group  means 
i=l, • • • ,t 
r=l, . . • ,p 


and  the  pooled  within  group  sums  of  cross-product  deviations 


g  ni 

w  =  E  Z  (x.  .  ~x.  )(x..  ~x^  ) 
rs  i=1  j=1  ijr  tr  ijs  is 


r=l, . • .  i p 
s=l,  -  •  •  ,P 


A  qxq  submatrix,  W^,  of  the  matrix,  W,  whose  ijth  entry  is  w.  .  is  defined 

by  a  stepwise  procedure.  The  stepwise  inclusion  of  variables  is  guided  hv 

the  F  approximation  to  Wilk's  A  statistic.  Let  A=W. and  let  (A)..=a.  . 

11  i.l  i  j 

The  squared  Mahalanobis  distance  of  sample  j  in  class  i  from  the  mean  of 


;roup  k  is  given  by 


D  ijk=(n-t)  Z  E  <x  -xkr 
J  r=l  s=l 


)ars^xiis'Xks) 


j=l, . . .ni 
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Finally  the  posterior  probability  that  sample  j  from  class  i  comes  from 
class  k  is  given  by 

pijk=pkexp(-1/2  Dijk  ^  prexp(-l/2D^.r) 

i=l . t 

j=i, . . . ,n. 
k=l, • • • ,g 


The  effectiveness  of  this  approach  is  dependent  on  the  assumptions  of  normalitv 
and  equivalence  of  the  covariance  matrices. 


4.1.2  Pooled  Covariance  Discrimination 


The  posterior  probabilities  of  a  sample  belonging  to  each  ol  several 
classes  can  be  used  to  rank  the  likelihoods  of  membership  in  each  of  the 
classes.  If  each  class  is  assumed  to  have  a  multivariate  normal  distribu¬ 
tion  and  all  classes  have  a  common  covariance  matrix,  then  the  estimation  of 
the  posterior  probabilities  is  considerably  simplified.  Software  for  the 
computation  of  these  probabilities  is  available  in  the  BMDP  package  [Dixon 
1979].  We  now  describe  the  computation  of  these  quantities.  For  further 
details,  see  [Dixon  1979].  Let  p  denote  the  number  of  variables,  g  the  num¬ 
ber  of  classes  used  for  design,  t  the  total  number  of  classes  and  x..  the 

i.l  r 

value  of  variable  r  of  sample  j  of  class  i.  Furthermore  let  n  be  the  total 
number  of  samples  and  let  n^  be  the  number  of  samples  in  class  i.  In  the 
experimental  work,  each  class  was  equally  divided  into  two  classes,  one  for 
designing  and  one  for  testing.  Thus  t=2g.  First  compute  the  group  means 


Kir  ■ 


i=l, . . . ,t 
r=l, . . . ,p 


and  the  pooled  within  group  sums  of  cross-product  deviations 

Ji  ji  r'‘ . . 

S=l,  •  ■  .  ,P 

A  qxq  submatrix,  W^,  of  the  matrix,  W,  whose  ijth  entry  is  w_  is  defined 

by  a  stepwise  procedure.  The  stepwise  inclusion  of  variables  is  guided  bv 

the  F  approximation  to  Wilk's  A  statistic.  Let  A=W  and  let  (A)..=a 

11  i.l  i  j 

The  squared  Mahalanobis  distance  of  sample  j  in  class  i  from  the  mean  of 
group  k  is  given  by 

’l, A  A^tr-V,ars<*iJs-*ks) 


r=l  s=l 


i=l , • • • , t 

.1  =  1,  •  •  -ni 

k=l , . . . ,g 
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4.1.3  Synthetic  Data  Experiments 


Simulation  studies  were  done  to  compare  the  performances  of  class  label  1 
using  pooled  covariance  density  estimation  and  disjoint  principal  components. 
The  purpose  of  these  studies  was  to  determine  which  of  these  algorithms  is 
more  suitable  for  providing  a  ranking  of  class  labels.  While  these  experi¬ 
ments  shed  some  light  on  the  characteristics  of  the  methods  involved,  no 
firm  guidelines  can  be  given  for  selecting  a  method  except  in  extreme  cases. 

Fifteen  simulations  were  performed.  The  data  set  for  each  simulation 
consisted  of  four  classes,  of  one-hundred  3-dimensional  vectors  each.  The 
means  of  the  four  classes  are  (0,0,0),  (1,0,0),  (0,1,0),  and  (0,0,1).  The 
f i f  aen  simulations  can  be  divided  into  five  groups  of  three  simulations  each 
Within  each  group  of  three  simulations  the  same  group  of  four  covariance 
matrices  was  used  for  each  of  the  three  trials,  but  different  sets  of  random 
vectors  were  generated  from  these  distributions.  None  of  the  covariance 
matrices  for  these  fifteen  trials  is  a  diagonal  matrix. 

For  each  simulation,  fifty  of  the  one-hundred  samples  from  each  class 
were  used  for  designing  decision  procedures  and  fifty  were  used  for  testing. 
For  each  of  the  200  test  samples,  twelve  numbers  were  calculated.  First  the 
true  density  function  of  each  class  was  used  to  generate  the  probability  of 
the  sample  coming  from  each  class.  The  probability  of  each  sample  belonging 
to  each  of  the  classes  was  then  estimated  using  the  pooled  sample  covariance 
matrix  over  all  classes.  Finally  the  distances  of  the  same  from  the  princi¬ 
pal  component  model  for  each  class  was  generated.  The  principal  component 
distances  were  converted  into  weights  in  the  following  way.  Let  d^,  d7,  d^, 
d^  be  the  four  distances  for  a  sample.  To  each  d^,  i=l,...,4,  we  correspond 

the  weight  w.=  — — — {-^1 - - -  i=l,...,4.  Before  computing  these 

1/d l/d2+  l/d3+  l/du, 

weights,  any  d^  such  that  d^<.0001  was  replaced  by  .001.  Using  this  nor¬ 
malization  procedure,  all  weights  lie  between  zero  and  one  and  larger  weights 


correspond  to  better  class  fits.  For  each  sample,  the  three  sets  of  four 
numbers  were  converted  to  rankings.  Thus  for  each  sample  and  each  of  the 
three  methods  of  assigning  weights  we  define  a  sequence  a^,  a-j,  of 

four  distinct  integers  chosen  from  the  set  {l,2,3,4}  as  follows.  Let  p  , 

P2»  p3»  P4  be  the  weights  or  probabilities  of  the  sample  belonging  to  classes 
1,  2,  3,  4  respectively.  Then  a^=j  if  pj  is  the  ith  largest  of  the  numbers 
Pl’  p2’  p3’  P4‘  F°r  eac^  s^mulation  the  confusion  matrix  for  each  method 
was  computed.  These  matrices  are  shown  in  Figure  4.1.  The  covariance 
matrices  are  given  in  Figure  4.2,  Since  the  class  ordering  given  by  the  true 
probabilities  is,  on  the  average,  optimal,  this  ordering  was  compared  with 
the  ordering  given  by  the  estimated  probabilities  and  disjoint  principal 
components.  In  Figure  4.3,  the  number  of  times  each  of  these  two  orderings 
agreed  with  the  ordering  determined  by  the  true  probabilities  for  the  top 
choice,  the  top  two  choices,  and  all  four  choices  is  given.  Finally  in  Figure 
4.4,  the  number  of  times  these  two  orderings  have  the  first  two  classes  correct 
or  reversed  is  given.  The  next  section  discusses  the  results. 
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Figure  4.1  Confusion  Matrices  for  fifteen  simulations  using  true 

probabilities,  estimated  probability  and  disjoint  principal 
components  for  classification.  Rows  correspond  to  true 
classes  and  columns  correspond  to  classification  results. 
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Figure  4.1  (cont) 


Estimated  Probabilities  Disjoint  Principal  Cbmponents 


(#  of 

classes  ranked 

correctly) 

(//  of 

cases  ranked 

correct  1 

Simulation  No. 
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62 

MEAN 

172.1 
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93.4 

157.9 

99.2 

55 

STANDARD  DEVIATION 

4.6 

16.0 

12.9 

8.4 

18.7 

18.3 

Note:  Aiy  sample  with  at  least  the  first  class  correct  is  counted  in 

column  one.  Any  sample  with  at  least  two  classes  correct  is 
counted  in  column  2. 


Figure  4.3  Number  of  correct  rankings  (e.g.  most  likely  class  is  correct,  two 

most  likely  classes  are  correctly  predicted  by  the  given  method)  for 
each  of  the  fifteen  simulation  experiments. 
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Disjoint  Principal 


Estimated  Probabilities 

ComDonent 

Simulation 

No. 

(Count) 

(Count) 

1 

119 

108 

2 

130 

118 

3 

126 

123 

4 

123 

118 

5 

123 

112 

6 

127 

118 

7 

143 

100 

8 

142 

88 

9 

143 

129 

10 

149 

154 

11 

175 

136 

12 

146 

145 

13 

123 

119 

14 

133 

136 

15 

140 

124 

Mean 

136.1 

121.9 

Standard  Deviation 

14.5 

16.9 

Figure  4.4 

Number 

of  times  first  two  classes  are  correct 

or  reversed  for 

each  of 

the  fifteen  simulation  experiments. 

137 


4.1.4  Discussion  of  Region  Weight  Evaluation 

The  simulation  experiments  indicated  no  advantage  of  disjoint  principal 
components  over  density  estimation.  No  theory  exists  for  predicting  the 
relative  performance  of  these  methods.  A  greater  variety  of  covariance 
matrices  should  be  examined  to  determine  conditions  under  which  each  method 
dominates.  Once  an  adequate  supply  of  examples  in  which  each  method  dominates 
is  collected,  an  attempt  should  be  made  to  relate  properties  of  the  distri¬ 
butions  to  the  success  of  the  various  methods.  One  possible  distribution 
property  is  the  similarity  of  the  individual  class  covariance  matrices  to 

the  pooled  covariances  matrix.  A  crude  but  simple  measure  of  this  distance 

n2 

may  be  obtained  by  viewing  an  nxn  covariance  matrix  as  a  point  in  R  .  The 
similarity  measure  can  be  taken  to  be  the  sum  of  the  distances  from  the  in¬ 
dividual  class  covariance  matrices  to  the  pooled  covariance  matrix.  A  second 
measure  of  covariance  matrix  similarity  may  be  obtained  by  comparing  the 
estimated  posterior  probability  of  each  sample  based  on  its  class  covariance 
matrix  and  the  estimated  probability  based  on  the  pooled  covariance  matrix. 

The  sums  of  the  differences  of  these  quantities  may  be  used  as  a  measure  of 
similarity.  Neither  of  the  above  measures  directly  takes  into  account  di¬ 
rectional  differences  in  the  structure  of  the  distributions.  A  simple  mea¬ 
sure  of  this  might  be  the  expected  angle  (over  all  classes)  between  the  first 
principal  directions  for  each  pair  of  classes,  where  by  principal  direction 
we  mean  the  eigenvector  corresponding  to  the  first  principal  component. 
Numerous  other  simple  measures  could  be  defined.  Correlations  between  these 
measures  and  the  success  of  one  method  over  another  could  then  serve  as  guide¬ 
lines  in  the  selection  of  an  appropriate  data  dependent  procedure  for  real  dat 
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4.1.5  Discission  of  Region  Label  Rankings 

The  comparison  of  methods  for  ranking  region  labels  is  difficult  to  per¬ 
form  on  image  data  since  the  only  standard  for  evaluation  is  where  the 'true 
class  is  located  in  the  rankings.  Definitions  of  class  similarity  could  be 
defined  in  terms  of  the  confusion  matrix  of  a  classification  procedure  such 
as  a  decision  tree.  Unfortunately,  the  evaluation  of  rankings  in  terms  of 
these  similarity  definitions  is  muddled  by  the  somewhat  arbitrary 
choice  of  a  standard  for  ranking.  The  disjoint  principal  components  (DPC) 
procedure  was  performed  on  a  test  set  of  one-hundred  and  twenty  samples  from 
the  aerial  image,  frame  4621,  supplied  by  USAETL.  These  samples  were  divided 
into  two  groups,  sixty  forest  samples  and  sixty  field  samples.  Each  sample 
consisted  of  the  thirty-two  rings  representing  ROSA  spatial  frequency  data. 

In  all  DPC  tests,  thirty  samples  from  each  class  were  used  for  the  de¬ 
sign  set  and  thirty  samples  for  the  test  set.  Due  to  the  small  number  of  sam¬ 
ples  used,  a  subset  of  the  thirty-two  rings  were  selected  for  application  of 
the  DPC  procedure.  For  this  subset  of  selected  features,  the  resulting  classifica¬ 
tion  are  given  in  Fig.  4.5.  Rings  1,  2,  3,  30,  31,  32  were  useful  as  a  start¬ 
ing  set  of  features  for  DPC.  In  addition,  the  density  estimation  procedure 
described  in  Section  4.1.2  was  applied  to  this  data.  For  this  work,  samples 
from  each  class  were  divided  into  two  groups  of  approximately  equal  size  using 
a  random  procedure  to  places  samples  in  design  or  test  sets.  Each  sample  had 
a  .5  probability  of  being  placed  in  the  design  or  the  test  set.  The  results 
of  this  computation  are  given  in  Figure  4.6.  The  histogram  of  the  first 
principal  component  of  the  samples,  using  the  pooled  covariance  matrix,  is  given 
in  Figure  4.7.  From  this  Figure  we  see  that  these  classes  separate  very  well 
using  the  first  principal  component  to  represent  the  data. 
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Forest  versus  Field  Discrimination 


Confusion  Matrix 

Density  Estimation  Using  a  Pooled  Covariance  Matrix 


Number  of  Cases  Classified  Into 
Group  Field  Forest 
Field  31  2 

Forest  0  25 


Percent  Correct 

93.3 

100 


Figure  4.5  Forest  versus  Field  Discrimination  (Density  Estimation). 


Forest  versus  Field  Discrimination 


Confusion  Matrix 

Disjoint  Principal  Components  (1  component) 

Number  of  Cases  Classified  Into  Percent  Correct 

Group  Field  Forest 
Field  29  1 

Forest  3  27 


Figure  4.6  Forest  versus  Field  Discrimination  (Disjoint  Principal  Components). 
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Legend:  Design  Set  Test  Set 


Figure  4.7  Histogram  of  the  First  Principal  Component  of  the  Pooled  Covariance  Matrix  for  the  Field, 
Forest  Data 


4 . 2  Overview  of  Region  Match.! ng 

The  classification  of  regions  in  an  image  is  a  difficult  task  which  some¬ 
times  requires  application  of  several  levels  of  preprocessing.  In  this  sec¬ 
tion  we  describe  two  procedures  for  taking  a  tentative  set  of  region  labels 
and  using  region  features  to  obtained  a  less  ambiguous  labelling.  In  addi¬ 
tion  we  describe  some  experiments  indicating  a  framework  for  a  detailed  study 
of  one  of  these  methods  using  real  image  data. 
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Region  image  matching  can  be  done  by  finding  a  set  of  features  which  de¬ 
scribe  the  regions  and  then  pairing  regions  which  have  the  best  matching  set 
of  features.  It  is  desirable  that  the  decisions  take  into  account  the  adja¬ 
cency  information  of  the  regions,  so  that  adjacent  regions  in  the  image  match 
adjacent  regions  in  the  map. 

One  such  method  of  region  matching  yields  a  measure  of  similarity  between 
pairs  of  regions,  one  from  each  of  the  two  images  to  be  matched  [Price  and 
Reddy  1979].  This  method  makes  no  assumptions  about  the  relative  displace¬ 
ment  and  orientation  of  the  pictures.  The  steps  of  their  algorithm  are  as 
follows : 

1)  Segment  the  image. 

2)  To  each  region  i  assign  a  set  V^,  ....  Vln  where  denotes  the 
value  of  the  j th  feature  for  region  i.  These  numbers  may  describe 
features  such  as  shape,  size,  position,  spectral  values,  etc.  To 
each  pair  of  regions,  region  i  from  image  1  and  region  j  from  image 
2,  define  the  region  to  region  match  rating,  R.^  ,  by: 

Rij  JD  Vik  ~  Vjk  |  Wk  Sk 

where  W^  is  a  normalization  factor  for  the  kc^  feature  and  is  a 
measure  of  importance  of  the  k1"^1  feature.  Larger  values  of  R^j 
indicate  good  matches. 

3)  In  this  step  we  attempt  to  improve  the  accuracy  of  the  rating 

by  taking  Into  account  adjacency  information.  To  each  region  R^  in 
image  1,  assign  the  region  Rj  in  image  2  which  maximizes  R^..  Let 
denote  the  set  of  regions  in  image  1  which  are  neighbors  of  region 
R^  and  have  a  match  in  image  2.  We  say  two  regions  are  neighbors  if 
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they  have  a  common  boundary  point.  Let  Nj  denote  those  regions  in 

Imago  2  which  match  the  regions  in  1^.  Let  N| '  denote  those  regions 

in  N.  which  are  neighbors  of  Rj  .  The  neighbor  f  eature  value  (N'KV)  ot 
J  J 

Rt  is  defined  to  be  the  number  of  elements  in  and  the  NFV  for  Rj 

is  the  number  of  elements  in  Nj  ’ .  Recompute  the  R^j's  using  this 
additional  feature  and  assign  to  each  region  in  image  1  the  region 
in  image  2  which  matches  it  best. 

This  matching  algorithm  is  designed  to  be  invariant  under  rotation  and 
translation  of  the  images.  By  omission  of  size  features,  the  algorithm  can 
be  made  invariant  to  scale  change.  Dissimilar  image  matching  and  image  to  map 
matching  can  be  handled  by  this  method. 
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4.2.2  Graph  Theoretical  Region  Matching 


This  section  deals  with  a  method  for  finding  corresponding  regions  in 
an  image  and  a  map.  The  procedure  [Pavlidis  1977]  described  takes  two  lists 
of  regions,  one  from  the  image  and  one  from  the  map,  together  with  a  set  of 
possible  matches  between  image  and  map  regions,  and  computes  a  region  cor¬ 
respondence  maximizing  the  number  of  regions  matched.  We  now  give  a  precise 
formulation  of  the  problem. 

Assume  the  map  consists  of  regions  and  the  image  consists  of 

regions  T^,...,T  .  Assume  to  each  map  region  we  assign  a  set  H(S^)  of 
image  regions.  Intuitively  the  image  regions  in  H(S^)  have  been  assigned  to 
because  they  may  match  it  on  the  basis  of  properties  such  as  site,  shape, 
texture,  etc.  Our  goal  is  to  find  a  subset  RcS  and  a  one-to-one  function  g: 
R-*T  satisfying  the  following  properties: 

1)  i/j=»g(Si)^g(Sj)  for  S±,  S j eR 

2)  VSl£R  g(S1)eH(Si) 

3)  Given  any  other  pair  (R',g')  satisfying  condition  (1)  and  (2),  |r'|<|r|, 
where  |  |  denotes  the  number  of  elements  in  the  set. 

The  computation  of  an  R  and  g  in  the  above  problem  can  be  solved  using  a 
method  from  network  flow  analysis.  Before  describing  the  procedure,  we  dis¬ 
cuss  the  reasons  for  studying  this  method.  The  labelling  of  regions,  in  an 
image  based  on  region  descriptors  such  as  ring  and  wedge  data  together  with 
structural  information,  such  as  the  types  of  regions  which  can  share  a  common 
boundary,  can  be  a  very  time  consuming  and  error  prone  task.  The  complexity 
of  labelling  algorithms  depends  upon  the  number  of  regions  to  be  labelled  and 
the  extent  of  ambiguity  present.  The  type  of  region  matching  presented  here 
can  be  viewed  as  a  fast  procedure  for  giving  a  rough  match  between  imago  and 
map  regions.  Both  individual  region  descriptions  and  inter-region  relation- 
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snips  can  be  included  in  the  matching  procedure.  The  matching  obtained  us  in? 
this  algorithm  can  then  serve  as  input  to  more  costly  algorithms  for  matching 
additional  regions  and  making  corrections  on  existing  matches. 

We  now  describe  [Bondy  1976]  the  matching  problem  in  graph-theoret i i  . 
terms  and  give  a  standard  method  for  solving  it. 


Def .  A  graph  G  is  a  pair  (V,E)  where  V  is  a  finite  set  {v^,...,vn}  and  the 
v^  are  called  nodes  or  vertices.  E  is  a  set  of  unordered  pairs  of 
elements  and  its  elements  are  called  edges.  If  e^Cu.v),  u,v  e  V  then 
we  may  say  e^  joins  u  and  v,  and  u  and  v  are  endpoints  of  e^,  and  u  and 
v  are  called  adjacent  vertices. 

Def.  A  subgraph  G*  of  a  graph  G=(V,E)  is  a  graph  (V',E')  such  that  V'cV, 
E'cE. 

Def.  An  edge  with  distinct  ends  is  called  a  link. 

Def.  Two  edges  are  called  adjacent  if  they  share  a  vertex. 

Def.  Let  G=(V,E)  be  a  graph.  A  subset  M  of  E  is  called  a  matching  if  G  if 
its  elements  are  links  and  no  two  elements  are  adjacent  in  G.  M  is 
called  a  maximum  matching  if  G  has  no  matching  M'  with  |m'|>|m|,  where 
|  j  denotes  the  number  of  edges. 

graph  maximal  matching 


Example  4.1  Maximal  matching 


Def .  A  bipartite  graph  G=(V,E)  is  a  graph  with  the  property  that  V  can  be 
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partitioned  into  two  sets  X  and  Y  such  that  each  edge  has  one  end  in 
X  and  one  end  in  Y. 


x={l,2,3} 


Example  4.2  Bipartite  Graph 


y={4,5} 


We  now  use  the  above  notation  to  reformulate  our  region  matching  prob-1 
lem.  Assume  S^,  1\,  and  H  are  defined  as  in  the  beginning  of  this  section. 
Define  a  graph  G=(V,E)  by  V=S  U  T,  E={ (S^ ,1\  ) | Tj eH(S^)  }.  G  is  a  bipartite  graph 
with  V  partitioned  into  S  and  T.  The  solutions,  (R,g),  to  our  original  prob¬ 
lem  can  be  put  into  a  one-to-one  correspondence  with  the  maximum  matchings  of 
the  bipartite  graph  G.  To  a  solution  (R,g)  we  correspond  the  edges  given  by 
the  graph  (in  the  analysis  sense)  of  g,  i.e.  (Sj,T.)  is  an  edge  iff  g(S.)=T4. 

J  x  J 

Conversely,  given  a  maximum  matching  M  of  G,  the  corresponding  solution  (R,g) 
is  given  by: 

g^KSj.Tj)  :  (S1,Tj)eM}  (viewing  the  function  g  as  a  set  of 
ordered  pairs) . 

R=  the  domain  of  g. 

We  now  describe  a  procedure  for  finding  a  maximum  matching  in  a  bipartite 
graph  which  will  have  two  additional  nodes,  c  and  b,  called  a  source  and  a 
sink  respectively,  and  js|  +  |t|  additional  edges.  In  addition  each  edge 
will  have  a  direction  and  a  number  associated  with  it.  The  graph  G'  will  have 
an  edge  joining  c  to  each  node  in  S.  Each  such  edge  will  have  weight  one  and 
will  be  directed  away  from  c.  In  addition,  G'  will  have  an  edge  from  each 
node  in  T  to  b.  Each  of  these  nodes  will  be  directed  toward  b  and  will  have 
unit  weight.  In  addition  each  edge  in  G'  joining  a  node  in  S  and  a  node  in  T 
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will  be  directed  from  S  to  T  and  will  have  weight  1. 

The  maximal  matching  can  now  be  posed  as  a  network  flow  problem.  A 
physical  analogy  is  to  represent  the  graph  by  a  set  of  pipes,  one  for  -a  . 
edge,  where  flow  occurs  only  in  the  specified  direction.  The  vertices  in  ' 
and  T  represent  connections  between  pipes.  Flow  can  occur  between  any  twr 
connected  pipes  subject  to  the  directionality  restriction.  The  numbers  as¬ 
signed  to  the  edges  represent  the  maximum  rate  of  flow  through  a  pipe.  Thus, 
if  a  pipe,  f,  with  a  flow  rate  n  is  connected  and  flowing  into  a  pipe,  g, 
with  flow  rate  m,  with  n>m,  and  with  no  other  pipes  connected  at  this  vertex, 
then  f  car  only  support  a  flow  rate  of  m  due  to  the  limit  imposed  by  g.  The 
source  c  is  assumed  capable  of  providing  flow  to  the  network  at  an  arbitrarily 
large  rate. 

It  can  be  shown  that  the  maximal  matching  of  a  bipartite  graph  can  be 
obtained  by  finding  the  flow  in  G'  maximizing  the  rate  of  flow  into  b.  This 
can  be  seen  intuitively  be  noting  that  the  restriction  to  all  flow  rates  being 
zero  or  one  in  G’  allows  us  to  consider  a  flow  of  one  in  an  edge  joining  S 
and  T  as  the  inclusion  of  this  edge  in  the  maximal  matching  and  a  flow  rate 
of  zero  as  excluding  this  edge  from  the  matching.  Thus  two  edges  from  S  to 
the  same  node  in  T  would  require  a  flow  rate  of  two  to  this  node,  but  the  flow 
rate  out  of  such  a  node  is  restricted  to  be  one.  Hence  this  situation  cannot 
occur  and  our  matching  is  one  to  one.  To  see  that  the  matching  is  maximal, 
note  that  the  flow  into  b  is  equal  to  the  sura  of  the  flow  rates  of  the  edges 
going  from  S  to  T,  which  is  in  turn  equal  to  the  number  of  edges  in  the  match¬ 
ing.  We  now  give  a  precise  description  of  the  network  flow  problem. 

Pgf •  A  directed  graph,  G,  on  a  set  V={v^, . . . ,vn)  is  a  subset,  E,  of  VxV  such 

that  for  each  (v^,  Vj)eE,  v^Vj .  We  call  the  elements  of  V,  the  vertices 
of  G  and  the  elements  of  E  the  edges  of  G.  We  say  the  (v^,Vj)eE  is  an 
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edge  from  the  vertex  v  to  the  vertex  Vy  Thus  a  directed  graph  may 
be  thought  of  as  an  ordinary  graph  in  which  each  edge  is  given  a 
direction. 

Def .  A  network  N  is  a  directed  graph  G  with  two  distinguished,  disjoint,  non¬ 
empty,  subsets  X  and  Y  of  vertices  of  G  and  a  non-negative  integer  valued 
function  d  defined  on  the  edges  of  G.  The  vertices  of  X  and  Y  are  called 
the  sources  and  sinks  of  N. 

In  the  notation  of  our  matching  problem  X={c}  and  Y={b},  i.e.,  ( X [ = | Y [ = 1 . 
The  set  of  vertices  of  G  belonging  to  neither  X  nor  Y  are  called  intermediate 
vertices  of  G,  and  is  denoted  by  I,  The  function  d  is  called  the  capacity 
function  of  N  and  the  value  of  this  capacity  function  on  an  edge  e  is  called 
the  capacity  of  e.  Let  V  and  E  denote  the  vertex  and  edge  sets  of  G.  For 
any  integer-valued  function  f  on  E,  and  any  vel,  define  f~(v)=£f(e)  where  the 
sum  is  taken  over  all  edges  in  E  of  the  form  (Vj ,v)  .  Similarly  we  define 
f+(v)=If(e)  where  the  sum  is  taken  over  all  edges  in  E  of  the  form  (v,^). 

Thus  f+(v)  represents  the  flow  out  of  v  and  f  (v)  represents  the  flow  into  v. 


Def.  A  flow  in  a  network  N  is  an  integer-valued  function  f  defined  on  A  such 
that 


and 


0  f(e)  c(e)  for  all  eeE 
f  (v)  =  f+(v)  for  all  vel. 


The  function  f  may  be  thought  of  as  the  assignment  of  a  feasible  flow 
rate  through  each  edge,  subject  to  the  restrictions  that  the  flow  is  limited 
by  the  capacity  of  an  edge  and  that  the  flows  into  and  out  of  an  intermediate 
vertex  must  be  equal.  From  this  point  onward  we  restrict  ourselves  to  the 
case  X={c}  and  Y*{b}.  We  define 
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Def .  The  value  of  a  flow,  f,  on  n  Is  £(f)=f+(c). 

Def.  A  flow,  f,  in  N  is  called  a  maximum  flow  if  there  is  no  flow  g  in  N  su' 
that  £ (g)  >  £(f). 

Various  algorithms  [Ford  1962,  Nijenhuis  1975]  for  finding  maximum  i iov. 

in  networks  have  appeared  in  print.  We  have  used  the  algorithm  given  in 

[Nijenhuis  1975]  for  our  experimentation.  For  the  flow  problem  arising  in 

maximal  matching,  with  n  regions,  i.e.  |s|  +  |t|  =  n,  and  m  edges,  the  com- 

1/2 

plexity  of  the  algorithm  is  0((n+2)  m) . 


ti 
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4. 2. 2.1  Region  Matching  Experiments 

Several  region  matching  experiments  were  performed  to  determine  the  use¬ 
fulness  of  various  features  in  performing  region  matching  between  two  images 
(or  an  image  and  a  map)  using  maximal  matching  in  a  bipartite  graph.  These 
experiments  were  performed  on  synthetic  region  data.  Each  artificial  image 
used  consisted  of  a  20x20  grid  subdivided  into  regions  but  no  other  information 
such  as  gray  levels  were  assumed.  Three  synthetic  images  A,  B,  and  C  shown  in 
Figure  4.8  were  used.  Experiments  were  performed  comparing  A  with  B  and  A  with 
C. 

The  first  experiment  was  an  attempt  to  determine  the  adequacy  of  crude 
region  locations  information  together  with  region  structure  information  for 
region  matching.  The  regions  were  labelled  as  shown.  Each  region  A^  in  A 
was  initially  matched  with  all  regions  in  B  which  had  a  boundary  point  within 
a  distance  of  four  from  at  least  one  boundary  point  of  the  region  A^  The  list 
of  tentative  matches  is  given  in  Table  4.1.  For  each  region  in  each  image,  five 
region  features  were  computed:  area,  perimeter,  area  divided  by  perimeter, 
maximum  horizontal  width  of  a  region  and  maximum  height  of  a  region.  The  fea¬ 
tures  for  the  regions  in  A,  B,  and  C  are  given  in  Table  4.2.  Not  all  the  fea¬ 

tures  had  to  be  used.  If  a  feature  was  to  be  used,  the  user  specified  a  toler¬ 
ance  a.  If  f^  is  the  value  of  a  feature  for  region  A^  and  f£  is  the  value  of 

the  same  feature  for  region  Bj  then  the  two  regions  fail  to  match  to  within 

the  tolerance  a  if  [ ^1^2 1  >  a.  The  results  of  the  experiments  performed  are 
given  in  Table  4.3.  Of  a  possible  ten  correct  matches  the  maximum  achieved 
was  four  and  in  this  case  four  incorrect  matches  were  also  made.  When  all  fea¬ 
tures  were  used  with  a  tolerance  of  .8,  only  three  correct  matches  were  made, 
but  no  incorrect  matchings  were  formed.  All  other  cases  had  three  or  fewer 
correct  matches  and  at  least  one  incorrect  match.  The  graph  matches  are  given 
in  Table  4.4. 
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(a)  Image  A 


(c)  Image 


Figure  4.8  The  artificial  i 
experiments. 
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Table 


Distance  4 


Region  A 
1 
2 

3 

4 

5 

6 

7 

8 
9 

10 


4.1  Feasible  Matching  of  image  B  to  . Lmage. .  A 


square  -  The  regions  can  match  if  their  boundaries  are  no  more 
than  4  squares  aparts  (in  the  space  of  distances  be¬ 
tween  curves) 


Region  B  matches 

1,2 

1,2, 4,5,6 

1.2.3.4.5.6.7.9.10 

2.4.3.7.8.9.10 

7.8.9.10 

10,11 

2.3.4.5.6.9.10 

1.2.3.4.5.6.7.9.10 

1.2.3.4.5.6.10 
2,3,4,5,6,7,8,9,10,11 
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Table  4.2 


Picture  A  Features 


Area 

Perimeter 

Area/Pei 

1 

6 

10 

.6 

9 

6 

10 

.6 

3 

30 

62 

.5 

4 

12 

14 

.8 

5 

18 

18 

1.0 

6 

24 

20 

1.2 

7 

9 

12 

.7 

8 

119 

56 

2.1 

9 

50 

30 

1.7 

10 

126 

66 

1.9 

Max  Hor. 
2 
2 

11 

4 

3 

4 
3 

10 

10 

9 


Max  Vert. 
3 
3 

20 

3 

6 

6 

3 

14 

5 

20 


Rec tan  . 

(  1,  5' 
(  1,10; 
(  5,10} 
(13,14) 
(19,17) 
(18,  3) 
(  9,  8) 
(  5,13) 
(  5,  3) 
(15,10) 


Picture  B  Features 


Area 

Perimeter 

Area/Perimeter 

Max  Hor . 

Max  Vert. 

Rectangle  C 
(  1,12) 

1 

16 

20 

.8 

2 

8 

(  5,13) 

2 

112 

52 

2.1 

10 

14 

(11,14) 

3 

11 

24 

.4 

1 

11 

(  9,  8) 

4 

15 

16 

.9 

5 

3 

(  5,  3) 

5 

16 

34 

.5 

11 

6 

(  5,  3) 

6 

50 

30 

1.7 

10 

5 

(14,18) 

7 

30 

22 

1.4 

6 

5 

(19,18) 

8 

15 

16 

.9 

3 

5 

(14,14) 

9 

18 

18 

1.0 

6 

3 

(16,  8) 

10 

102 

48 

2.1 

9 

15 

(19,  3) 

11 

15 

16 

.9 

3 

5 

Picture  C  Features 


Area 

Perimeter 

Area /Perimeter 

Max  Hi 

1 

15 

16 

.9 

3 

2 

125 

66 

1.9 

9 

3 

16 

16 

1.0 

4 

4 

24 

20 

1.2 

4 

5 

30 

62 

.5 

11 

6 

6 

10 

.6 

3 

7 

44 

30 

1.5 

10 

8 

12 

14 

.8 

4 

9 

108 

58 

1.9 

10 

10 

12 

14 

.8 

3 

11 

8 

12 

.7 

2 

Max  Vert. 

Rectangle 

5 

(19,  3) 

20 

(16,10) 

4 

(13,13) 

6 

(18,17) 

20 

(  5,10) 

2 

(  2,  1) 

5 

(  i) 

3 

(  8.  8) 

14 

(  5,13) 

4 

(  2,  9) 

4 

(  1,15) 
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Table  4.3  Experiments  in  Matching  Image  B  to  Image  A  using  the  vectors  4  squar. 
Feasible  Matchings  from  Table  4.1. 


Feature  codes 

1.  frea  of  region 

2.  Perimeter  ol  region 

3.  Area/Perimeter  ratio 

4.  Maximum  horizontal  extent  of  region 

5.  Maximum  vertical  extent  of  region 


Trial  1 

Matched  Regions 

Features  Used 

Tolerance 

Image  A 

Image  B 

Correctness 

4 

.3 

2 

6 

Wrong 

5 

.3 

3 

5 

R  ight 

6 

11 

R  ight 

10 

10 

Right 

Trial  2 
Features  Used 

Tolerance 

Image  A 

Matched  Regions 
Image  B 

Cbrrectness 

1 

.8 

3 

6 

Wrong 

2 

.8 

4 

4 

Wrong 

3 

.8 

6 

11 

Right 

4 

.8 

8 

7 

Wrong 

5 

.8 

10 

10 

Right 

Trial  3 

Matched  Regions 

Features  Used 

Tolerance 

Image  A 

Image  B 

Correctness 

1 

.5 

3 

5 

R  ight 

2 

.5 

6 

11 

Right 

3 

.5 

10 

10 

R  ight 

4 

.5 

5 

.5 

of  Match 


of  Match 


of  Match 


Trial  4 
Features  Used 
1 
3 


Tolerance 

.5 

.5 


Image  A 

3 

4 
6 

10 


Matched  Regions 
Image  B 
5 
4 

11 

10 


Cbrrectness 
R  ight 
Wrong 
R  ight 
Right 


of  Mrtch 


Trial  5 

Features  Used  Tolerance 
1  .5 


Image  A 

3 

4 

5 

6 

7 

8 
9 

10 


Matched  Regions 
Image  B 

7 

8 
9 

11 
3 
2 
6 

10 


Cbrrectness 
Wrong 
Wrong 
Wrong 
Right 
Wrong 
Right 
Right 
R  ight 


of  Match 
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Table  4.4 

True  matches 

of  linage  A  Regions 

with  Regions  from  Image  B 

and  C. 

linage 

A 

I mage  B 

1  magi'  C 

1 

1 

11 

2 

1 

10 

3 

3,5 

5 

4 

9 

3 

5 

8 

4 

6 

11 

1 

7 

4 

8 

8 

2 

9 

9 

6 

7,6 

10 

10,7 

2 
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la  the  second  experiment,  the  original  criterion  for  arriving  at  a  tenta¬ 
tive  match  was  modified.  For  each  region  a  center  was  computed.  The  center 
was  defined  to  be  the  point  with  coordinates  (x,y)  where  x  is  the  average  of 
the  leftmost  and  rightmost  horizontal  coordinates  of  the  region  (truncated  to 
an  integer)  and  y  is  the  average  of  the  topmost  and  bottom  most  y  coordinates 
of  the  region  (also  truncated).  Two  regions  are  tentatively  matched  if  their 
centers  are  no  more  than  four  units  apart.  Of  the  ten  regions  in  image  B  which 
have  at  least  one  tentative  match  in  region  A,  only  eight  correct  matches  can 
be  made.  In  matching  C  and  A  there  are  eleven  possible  correct  matches  but 
only  ten  can  be  determined  since  we  are  seeking  a  one  to  one  matching.  In 
matching  A  to  B  the  maximal  matching,  using  no  features,  is  given  in  Figure  4.9. 
This  matching  achieves  seven  matches  out  of  the  maximum  of  eight  possible  matches 
under  the  distances  between  center  restriction.  The  matching  of  A  to  C,  using 
no  features,  is  given  in  Figure  4.10.  In  this  matching  eight  of  the  ten  pos¬ 
sible  correct  matches  were  obtained. 
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Image  B 

• 

Possible  Image  A 

Maximal  Matching 

Correct 

Regions 

R  egions 

Image  A  Regions 

Mat  c.  In 

1 

1,2 

I 

X 

2 

8 

8 

X 

3 

none 

none 

4 

7 

7 

X 

5 

none 

none 

6 

9 

9 

X 

7 

none 

none 

8 

5 

5 

X 

9 

4 

4 

X 

10 

10 

10 

X 

11 

6 

6 

X 

Figure  4.9  Matching  of  Image  A  to  Image  B  where  possible  matches  assume 
the  centers  of  corresponding  regions  are  no  more  than  tour 
units  apart. 


Image  C  Possible  Image  A  Maximal  Matching  Correct 

Regions  Regions  Image  A  Regions  Matches 


1 

2 

3 

4 

5 

7 

8 
9 

10 

11 


6 

10 

4,10 

5 

2.3 
9 
9 
7 

8.3 

2.3 
1 


6 

10 

4 

5 
2 
9 


none 

7 

8 
3 


1 


x 

x 

X 

X 


X 

X 

X 


Figure  4.10  Matching  of  Image  A  to  Image  C  where  possible  matches 

assume  the  centers  of  corresponding  regions  are  no  more 
than  four  units  apart. 
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4. 2. 2. 2  Graph  Matching  Discussion 

The  evaluation  of  graph  matching  algorithms  for  region  labelling  is  a  com 
plicated  problem  due  to  the  large  number  of  variables  involved.  Maximum  mat.’ 
ing  in  bipartite  graphs  has  serious  limitations  such  as  the  inability  to  account 
for  both  fragmentation  and  coalescing  of  regions.  Since  the  matching  is  one  ii 
one,  a  region  which  becomes  fragmented  can  be  matched  to  at  most  one  of  its 
fragments.  One  possible  way  to  avoid  this  problem  is  to  redefine  the  problem 
by  changing  the  weights  on  the  network  graph  edges.  Suppose  we  have  two  images, 
A  and  B  and  that  the  flow  is  from  the  source  to  image  A  regions  to  image  B  regi-'i 
to  the  sink.  If  each  edge  from  an  image  A  region,  A^  to  an  image  B  region,  b^, 
is  assigned  the  area  of  A^  as  a  capacity  and  the  edge  from  B^  to  the  sink  is  as¬ 
signed  the  area  of  Bj  as  a  capacity  then  the  maximal  flow  algorithm  allows  for 
multiple  matchings  of  regions  subject  to  the  constraint  that  the  total  area  of 

all  regions  in  A  matching  with  B.  is  no  greater  than  the  area  of  B..  This 

-*  J 

change  in  the  algorithm  increases  the  complexity  of  the  algorithm  as  well  as 
introducing  other  complexities  in  interpretation.  An  alternate  approach  to 
eliminating  the  effects  of  region  fragmentation  is  to  perform  region  merging 
as  a  preprocessing  operation.  Selecting  adequate  merging  criteria  can  be  ex¬ 
pensive  and  offsets  the  advantages  of  graph  matching. 

The  selection  of  criteria  for  the  tentative  region  matches  for  creating 
the  bipartite  graph  should  be  based  on  extensive  analysis  of  image  data.  The 
reliability  of  region  features  such  as  shape,  the  approximate  location  of  region 
centers,  ring  and  vedge  data,  etc.  must  be  determined  for  a  large  collection  of 
regions.  Hand  matching  of  regions  in  images  taken  at  different  times  or  match¬ 
ing  of  images  to  maps  or  region  data  bases  can  provide  information  on  the  vari¬ 
ability  of  these  features.  If  the  variation  of  features  is  not  great  then  re¬ 
quirements  for  matching  can  be  made  stringent,  though  this  necessitates  high 
quality  region  feature  extraction. 
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•> . 0  Verification  of  Lineal  Features 

In  the  process  of  verifying  lineal  features,  it  is  assumed  t ' 
enough  information  has  been  extracted  from  an  image  so  that  hypotheses 
about  the  remaining  image  content  can  be  made.  In  the  verification  of  the 
existence  of  a  particular  lineal  feature,  the  rough  location  and 
orientation  of  the  feature  is  known.  That  is,  if  the  feature  exists  at 
all,  the  model  being  used  should  predict  approximately  where  the  feature 
is  with  respect  to  previously  detected  features,  how  long  it  is,  what 
its  shape  is,  etc.  For  example,  if  features  resembling  the  two  wings 
of  an  airplane  have  been  detected,  there  are  at  most  two  places  to 
search  for  the  tail.  Finding  the  hypothesized  feature  greatly  increases 
the  confidence  in  the  model  that  generated  the  hypothesis,  while  failure 
to  detect  the  predicted  feature  has  just  the  reverse  effect. 

Because  verification  is  done  with  model  prediction,  focused 
searches  can  be  performed.  Not  only  is  the  area  of  imagery  to  be  searched 
well-confined,  but  there  are  also  tight  constraints  on  shape  and  orientation. 
Thus  faint  or  hard  to  detect  features  can  be  found  more  reliably  and  more 
efficiently  in  the  top-down  mode  than  in  the  bottom-up  mode. 

The  technique  that  has  been  used  views  the  feature  as  a  set  of 
high  gradient  points  that  must  be  found  in  the  image.  The  transformation 
of  the  image  to  the  model,  obtained  by  running  the  registration  software, 
is  instrumental  in  the  verification  process.  The  inverse  of  this  trans¬ 
formation  is  used  to  predict  where  the  undetected  feature  should  exist. 

The  verification  software  then  proceeds  to  determine  whether  the 
hypothesized  feature  actually  exists. 
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>.  1  Technique  Used  In  Verification  Process 

'3.1.1  Profile  Search 

As  an  example  of  the  verification  process  consider  the 
verification  of  the  existence  of  Sherman  Creek  in  the  imagery  of 
Harrisburg,  Pennsylvania,  as  presented  in  Figure  3.2.  it  is  easy  to 

store  the  path  of  the  stream  as  an  ordered  set  of  points  in  some  coor¬ 
dinate  system,  as  is  done  in  cartographic  data  bases.  Given  a  registration  of 
the  imagery  containing  the  creek  to  the  cartographic  coordinate  system, 
it  is  easy  to  transform  the  points  of  the  mapped  feature  to  the  points 
(pixels)  of  the  image  where  the  feature  should  be  found.  Due  to  noise, 
distortion,  approximation  in  the  registration  transformation,  and  actual 
change  in  the  stream,  it  is  unlikely  that  the  feature  points  will  be 
found  exactly  where  they  are  predicted  to  be.  In  an  effort  to  find 
exactly  where  the  feature  point  does  lie,  a  technique  of  "profile  searching" 
is  used. 

In  the  profile  search,  a  search  for  the  feature  point  is  made 
along  a  "profile"  that  extends  from  the  predicted  point,  perpendicular  to 
the  direction  of  the  feature,  for  a  distance  defined  by  the  user  specified 
tolerance.  The  search  along  this  profile  begins  at  the  predicted  point, 
and  proceeds  outward  in  both  directions.  The  search  continues  in  both 
directions  until  a  "peak  point"  is  found.  A  peak  point  is  defined  as  a 
point  that  possesses  the  same  gradient  direction  as  the  predicted  point  and 
has  a  gradient  magnitude  that  is  above  the  user  supplied  threshold  and  is 
greater  than  the  next  point  along  the  profile.  This  technique  will  be 
further  clarified  through  the  use  of  an  example. 
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In  the  example  presented  In  Figure  ’•1,  the  predicted  feature 
point  was  at  location  (7232,4045)  in  stage  coordinates.  The  window 
sampling  the  image  is  a  32  pixel  x  32  pixel  array,  centered  at  (16,16). 

The  end  points  of  the  profile  are  dependent  on  the  user  supplied  tolerance 
value  and  the  direction  of  the  feature  gradient  at  the  predicted  point. 

For  this  point,  the  profile  extends  from  (27,20)  to  (5,12)  in  pixels. 

The  gradient  magnitude  is  calculated  at  each  of  these  points  using  the 
direction  of  the  gradient  at  the  predicted  point.  These  values  are 
presented  on  the  line  with  MAG  at  the  left.  The  user  supplied  threshold 
in  this  case  was  4.  The  search  begins  at  the  point  (16,16)  and  proceeds 
outward  in  both  the  direction  to  the  right  and  to  the  left.  The  threshold 
value  is  exceeded  for  the  first  time  at  point  (13,15)  where  the  gradient 
magnitude  is  27.  At  this  point  in  the  searching  process,  the  left  search 
is  at  point  (19,17)  whose  magnitude  is  below  threshold.  The  search  con¬ 
tinues  In  both  directions.  At  point  (12,14)  the  gradient  magnitude 
increases  to  38  and  at  point  (20,17)  the  gradient  magnitude  is  still 
below  threshold.  At  point  (11,14)  the  value  of  the  gradient  magnitude 
decreases,  indicating  that  a  peak  point  has  been  detected.  Since  the 
left  search  values  are  still  below  threshold,  the  algorithm  declares  that 
the  actual  feature  point,  in  terms  of  pixel  coordinates,  lies  at  point 
(12,14)  instead  of  point  (16,16). 

5.1.2  Quality  Evaluation 

When  the  verification  process  has  been  completed  for  a  specified 
lineal  feature,  an  evaluation  must  be  made  as  to  how  far  the  detected 


feature  points  deviated  from  their  predicted  position.  A  root-mean- 


STAGE  FOCUSED  AT  7232,4045 


square  value  is  one  measure  of  the  match.  Let  P  =  , . . . ,Pm, . . . ,Pn 

be  the  feature  point  set  in  map  coordinates  and  let  TJ,  be  the  transforms 
registering  the  imagery  to  the  map  coordinate  system.  Let  qffi  be  the  best 
detection  of  point  Pm  along  the  profile  passing  through  Then 

one  measure  of  the  verification  of  the  point  set  P  is 

D(p>  -  «JX  <J2(T;1(Pm>.qm))/N)1/2 

o 

where  d  (  «  ,  •  )  is  the  squared  distance  between  two  image  points. 

This  measure  is  only  taken  over  those  points  which  have  a  point  of 
detection  on  their  profile.  So  in  actuality  two  measures  of  quality  must 
be  used.  The  first  is  the  D(P)  above  and  the  second  is  the  percentage  of 
unmatched  points  that  are  present  for  that  feature. 

5.  1.  3  Servoeing 

As  was  stated  earlier,  due  to  several  factors,  it  is  unlikely 
that  the  feature  points  will  be  exactly  where  they  are  predicted  to  be. 
In  the  experiments  run,  this  was  found  to  be  true.  In  several  cases,  the 
points  of  the  feature  were  consistently  removed  from  the  predicted 
values  by  some  fixed  amount.  In  order  to  track  the  actual  feature 
better  and  to  overcome  the  inaccurracies  of  the  prediction  a  "servoeing" 
mechanism  was  instituted.  The  servoeing  mechanism  uses  information  from 
previous  points  to  make  adjustments  to  the  predicted  point.  The  method 
used  keeps  track  of  the  differences  between  the  predicted  point  and  the 
detected  maximum  on  the  profile  for  as  many  as  the  last  5  points.  THe 
differences  in  the  X-direction  and  the  Y-direction  are  averaged  and  are 
added  to  the  predicted  coordinates  to  produce  a  set  of  "adjusted"point 
coordinates. 


*  . 
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This  technique  has  been  implemented  and  tested.  With  servoeing, 
the  tracking  of  the  features  has  greatly  improved.  The  adjusted  predicted 
points  are  consistently  as  close  or  closer  to  the  actual  peak  points 
than  the  original  predicted  points  were.  The  servoeing  also  helps  to 
overcome  the  "warping"  effect  that  is  present  in  most  aerial  photographs. 
Because  this  warping  is  present,  the  registration  transformation  cannot 
be  accurate  over  the  entire  picture.  Therefore,  there  are  features 
present  in  the  photograph  that  may  not  be  accurately  verified  if  only 
the  inverse  of  the  registration  transformation  is  used.  The  servoeing 
technique  makes  the  minor  adjustments  that  are  needed  to  overcome 
these  inherent  inaccuracies. 
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5 . 2  Examples  from  4621 

The  verification  software  was  run  on  several  different 
photographs,  the  first  of  which  was  the  photograph  called  "4621",  in 
Figure  1.1.  As  can  be  seen,  most  of  the  features  of  this  photograph 
are  straight  lines,  some  very  prominent  and  others  not  very  clear  at 
all. 

The  registration  procedure  had  been  run  on  a  map  and  image 
pair  using  this  scene.  The  resulting  transformation  was  (308°, 578,  -1397). 
Using  this  transformation  as  a  basis,  an  inverse  transformation  was 
developed  for  use  in  the  verification  process.  The  inverse  transformation 
was  (52°,  -578,  1397).  The  verification  process  used  this  inverse  trans¬ 
formation  to  verify  the  existence  of  a  set  of  lineal  features  in  the 
image.  Some  of  the  features  verified  were  quite  distinct,  while  others, 
such  as  the  feature  from  point  2  to  point  27,  were  less  prominent. 

The  results  obtained  from  running  the  verification  on  this 
image  were  extremely  encouraging.  As  can  be  seen  in  Table  5.2.1,  the 
percentage  of  matched  points  for  every  one  of  the  lines  verified  was 
100%.  The  values  of  the  distance  function  were  also  small  for  all  of 
the  cases. 

As  examples  of  the  verifier  s  ''  ess,  output  from  a  run 

that  verified  two  of  the  features  in  4621  is  shown  in  Figure  5-2.  The 
first  feature  verified  is  the  one  running  from  point  //I  to  point  #29. 

This  is  a  prominent  road  that,  as  can  be  seen  from  the  presented  re¬ 
sults  possesses  strong  gradient  values.  For  each  predicted  point  a  pro¬ 
file  was  constructed  and  a  search  was  made  along  the  profile  for  the  peak 
value.  The  location  of  this  peak  value  was  recorded  by  the  servoeing 


Li 
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Table  5.2.1 


Results  of  Verification,  of  Features  in  4621 


Begin  Point  End  Point  Total  // 
of  Line  of  Line  of  Points 

Checked 


//  of  %  of 

Unmatched  Matched 
Points  Points 


1 

29 

41 

0 

100% 

2 

27 

21 

0 

100% 

1 

29 

41 

0 

100% 

8 

21 

41 

0 

100% 

3 

14 

41 

0 

100% 

16 

15 

21 

0 

100% 

12 

11 

21 

0 

100% 

3 

28 

11 

0 

100% 

Distance 
D(P)  in 
Stagels 

8.9 

19.1 

10.3 
4.7 

11.7 

21.4 

34.8 

16.2 
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Figure  3.2  Verification  results  of  feature  from  point  //I  to 
point  #29  of  4621,  with  use  of  servoeing. 
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mechanism  to  be  used  in  adjusting  the  next  predicted  point. 

In  Figure  5.2,  it  can  be  seen  that  the  first  predicted  point  was  at 
(5018,4990)  of  the  image.  The  actual  peak  was  not  at  the  predicted 
point,  which  is  (16,16)  in  window  coordinates,  but  at  (14,15).  In 
terms  of  stage  coordinates,  this  means  a  difference  of  -16  in  the  X- 
direction  and  -8  in  the  Y -direction.  This  is  taken  into  account  in  the 
calculation  of  the  next  predicted  point.  As  can  be  seen,  the  next 
transformed  point  in  the  image  is  at  (4963,5103),  but,  when  the  servoeing 
is  done  the  adjusted  point  becomes  (4955,5099).  The  result  of  the  profile 
search  gives  a  peak  value  of  117  at  window  coordinates  (15,15). 

Again  the  actual  peak  is  not  at  the  adjusted  predicted  point  but  it 

is  getting  closer.  Forthe  next  38  points  predicted  and  adjusted  for  this  feature , 

the  actual  peak  value  lies  either  at  (16,16),  dead  center,  or  (15,15) 

just  slightly  removed.  These  are  excellent  results  and  much  better 

than  those  obtained  without  use  of  the  servoeing  mechanism. 

Verification  of  the  same  feature,  from  point  #1  to  point 
#29,  without  the  use  of  servoeing  is  presented  in  Figure  5.3.  In  this 
case,  for  most  of  the  points  the  actual  peak  value  was  found  at  either 
window  coordinates  (13,15)  or  (14,15).  These  values  are  consistent 
and  close  to  the  center,  so  it  is  known  that  the  feature  is  present 
but  the  process  does  not  home  in  on  it.  In  fact,  it  was  this  type  of 
result  that  led  to  the  creation  of  the  servoeing  mechanism.  Because 
of  transformation  approximations,  picture  warping  or  other  factors,  the 
predicted  points  in  this  case  are  consistently  off  by  a  small  amount.  The 
servoeing  mechanism  adjusts  the  tracking  by  this  small  amount  and 
homes  in  on  the  detected  feature. 
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Figure  5 . 3  Verification  results  of  feature  from  point  #1  to 
point  #29  of  4621,  without  use  of  servoeing. 


The  feature  tracked  above  represents  a  very  prominent  roadway. 

In  order  to  check  out  the  verification  process  it  was  also  necessary  to 
attempt  the  verification  of  features  that  were  not  as  prominent. 

One  of  these  less  prominent  features  is  the  driveway  extending  from 
point  # 2  to  point  //27.  The  results  of  this  verification  are  presented 
in  Figure  54.  (Note  the  peak  detection  algorithm  used  in  this 
example  is  not  the  most  recent  scheme.  The  scheme  used  here  chose 
the  maximum  value  on  the  profile  regardless  of  where  it  was  located. 

It  was  found  that  this  technique  was  not  as  good  as  the  one  explained 
in  Section  5.1.3.)  Even  though  this  feature  is  not  nearly  as  prominent 
as  the  one  discussed  above,  the  procedure  was  able  to  verify  its 
existence  to  a  high  degree  of  confidence.  Refering  to  Table  5.2.1, 
it  can  be  seen  that  21  out  of  21  predicted  points  had  a  match  on  their 
profiles. 

The  results  of  the  experiment  run  on  4621  were  most  encouraging. 
For  every  feature  chosen,  a  100%  match  was  obtained  with  low  distance 
values.  However,  this  experiment  only  dealt  with  relatively  distinct 
features  that  were  all  straight  lines.  So,  though  the  results  were  very 
itisfying  a  more  difficult  set  of  features  was  required  to  continue 
the  testing  process.  This  more  challenging  set  was  obtained  from  the 
iireux,  France  images. 
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Schematic  of  features  used  in  verification 
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Results  of  verification  of  feature  from  point  K  to 
Pf^2)  °  13,  with  servoeing.  (part  2 


ire  quite  high.  The  two  sides  of  the  road  are  evident  in  the  profile 
by  the  high  positive  values  adjacent  to  the  low  negative  values.  The 
verifying  of  the  line  procedes  quite  accurately  even  around  the  shapp 
curve  in  the  road.  The  detected  peak  is  always  no  further  than  2 
pixels  off  the  center,  thus  resulting  ii.  the  extremely  low  value  for 
D{P). 

The  results  obtained  were  not  always  as  good  as  in  the 
above  case.  The  feature  extending  from  point  P  to  point  R,  is  a  very 
faint  road  that  runs  into  a  group  of  buildings  near  point  R.  The 
results  of  the  verification  of  this  road  are  presented  in  Figure  5.7. 

The  magnitude  values  for  the  gradient  in  the  area  of  this  line  are  low. 

For  many  of  the  profiles  the  threshold  value  is  never  exceeded,  therefore 
resulting  in  unmatched  points.  However,  on  closer  examination  there 
are  definitive  observable  peaks  in  the  profiles  but,  the  peak  values 
are  lower  than  the  user  supplied  threshold  of  4.  Also  when  predicted 
points  close  to  point  R  are  examined,  large  gradient  values  appear  at 
a  significant  distance  from  the  predicted  point.  These  large  values 
are  the  result  of  the  profile  passing  through  the  area  where  the  buildings 
are.  So  though  the  results  are  not  as  good  as  with  some  of  the  other 
features,  a  human  examination  of  the  area  can  explain  some  of  the  reasons 
for  this. 

An  experiment  was  also  run  to  try  to  verify  the  existence  of 
a  non-existent  feature  running  from  point  Q  to  point  T.  To  accomplish 
this,  a  set  of  points  was  taken  along  a  ficticious  feature  starting  at 
point  Q  and  ending  at  point  T.  This  set  of  points  was  then  presented 
to  the  verification  procedure.  Though  the  procedure  was  able  to  match 
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Figure  5.7  Results  of  verification  of  feature  from  point 
P  to  point  R  of  DREUX  13,  with  servoeing. 
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72%  of  the  points,  the  value  for  D(P)  is  much  greater  than  those  obtained 
for  any  of  the  other  features.  Upon  examining  the  output  produced  (see 
Figure  5.8),  it  is  noted  that  there  is  no  recognizable  pattern  to  the 
location  of  the  peaks.  This  leads  to  the  high  value  of  D(P)  and  decrease* 
ones  confidence  that  the  features  exists  in  the  image,  and  in  fact  it  does 
not. 

The  results  obtained  in  running  the  verification  software  on 
the  DREUX  13  image  have  buoyed  our  confidence  in  its  ability  to  verify 
features  in  an  image.  It  demonstrated  the  ability  to  follow  highly  curved 
lines,  find  both  distinct  and  not  so  distinct  features  and  indicate  fail¬ 
ure  when  an  attempt  is  made  to  track  a  feature  that  is  not  present. 


180 


<  3A2 ) 


GIVE  FILE  NAME  OF  INPUT  FILE  - 
CTDRQT 

INPU1  POINT,  X  ,  Y  , ANGLE*  13096  918 

TRANSFORMED  POINT  X,Y,G(J>  -  84C 

Q  ft  EC  IS-NODE  , DFLX , DELY  .LINK*  1 

ADJUSTED  X,Y-  8404  7745 

STAGE  FOCUSED  AT  8404,  7745 

J, ANGLE, SLQPE.K  »  1  4 . 7122B03 . 561 

XTAIL, YTAIL ,XHEAD ,YHEAD  -  16 

XTAIl  16  16  16  16  16  16  16  1 

YTAIL  25  24  23  22  21  20  19  1 

DIR  9  9  9  9  9  9  9 

MAG  -1  -i  -1  0  1  1  0  - 

MAX.  GRAD  FOR  POINT  •  1  WAS  1  AT 

DIST ,TOTPTS,UMCHPT  =  .00  1 


16  16 
17  16 


16 

16  16 


16  16 
13  12 

9  9 

1  0 


transformed  point 


Q  MFC 

IS-NODE, DELX, 

DEL  Y ,L I NK* 

1 

46 

-93 

2 

Q  REC 

IS-NODE, DEI  X  ,  DFLY ,L INK* 

2 

16 

-60 

3 

Q  REC 

IS-NODE  ,DFL.X,DFLY  ,1  INK* 

3 

-15 

-3? 

4 

Q  REC 

IS-NODf  , DEI  X , DFLY , LI NK  = 

4 

58 

-1  06 

5 

Q  REC 

IS-NODE, DF LX, DELY, LINK* 

5 

67 

-118 

1 

ADJUSTFD  X , Y-  10960 

8789 

STAGE 

FOCUSED  AT  10968,  8709 

J  ,  ANGLE,  SI.  OPE,  K  - 

t  s.ios 

-2.417 

38 

XTAtL, 

. YTAIL, XHEAD, YHEAD  «  1 

2 

27 

20 

5 

XTAIL 

12  13  13 

13  14  14 

14 

15 

15 

15 

16 

16 

17 

17 

YTAIL 

27  26  25 

24  23  22 

21 

20 

19 

10 

17 

16 

15 

14 

DIR 

7  7  7 

7  7  7 

7 

7 

7 

7 

7 

7 

7 

7 

MAG 

1  1  2 

1  0  1 

2 

2 

1 

0 

1 

2 

2 

1 

MAX.  GRAD  FOR  POINT  #23  WAS  56  AT  19, 
DIST .TOTPTS, UMCHPT  «  676.02  23  7 

INPUT  POINT,  X ,  Y  ,  ANGLE5*  16068  11124 

TRANSFORMED  POINT  X,Y,G(J)  -  11070 

Q  REG  IS-NODE ,DELX , DELY , LINK®  2  50 

Q  REC  IS-NODE, DELX, DELY, LINK-  3  -15 

Q  REC  IS-NODE, DELX, DELY, LINK*  4  58 

Q  REC  IS-NODE, DELX, DELY, LINK-  5  67 

Q  REC  IS-NODE, DELX, DELY, LINK-  1  46 

ADJUSTED  X,Y«  11112  8820 

STAGE  FOCUSED  AT  11112,  BB20 


19 

19 

19 

20 

20 

9 

R 

7 

6 

5 

7 

7 

7 

7 

7 

21 

45 

56 

12 

-29 

J, ANGLE, SLOP E.K 

m 

1  s. 

10S  - 

-2  -  ■ 

417 

38 

XTAIL, YTAIL 

,  XHEAD , YHEAD  - 

12 

27 

20 

5 

XTAIL  12 

13 

13 

13  14 

14 

14 

15 

IS 

IS 

16 

16 

17 

17 

17 

1R 

IB 

18 

19 

19 

19 

20 

20 

YTAIL  27 

26 

25 

24  23 

22 

21 

20 

19 

18 

17 

16 

15 

14 

13 

12 

11 

10 

9 

8 

7 

6 

5 

DIR  7 

7 

7 

7  7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

MAG  3 

1 

1 

0  0 

0 

2 

5 

4 

a 

6 

25 

42 

24 

-12 

-43 

-28 

-7 

5 

4 

1 

-1 

0 

MAX.  GRAD  FOR  POINT 

•  24  WAS  42 

AT 

17 

15 

DIST , TOTPTB, UMCHPT  -  607.34 

INPUT  POINT,  X,Y, ANGLE-  16224 
TRANSFORMED  POINT  X,Y,G(J)  - 
Q  REC  IB-NODE, DELX, DELY, LINK- 
Q  REC  IS-NODE, DELX, DELY, LINK- 
Q  REC  IS-NODE, DELX, DELY, LINK- 
Q  REC  IS-NODE, DFLX, DELY, LINK- 
Q  REC  IS-NODE, DELX, DELY, LINK- 
ADJUSTED  X , Y-  11319  8896 

STAGE  FOCUSED  AT  11319,  8896 


24 

11270 

11264 


J, ANGLE, SLOPE, K 

•  1  5, 

105  - 

2. 

417 

38 

XTAIL, YTAIL, 

.XHEAD, YHEAD  « 

12 

27 

20 

5 

XTAIL  12 

13 

13  13  14 

14 

14 

15 

15 

15 

16 

16 

17 

17 

17 

IB 

IB 

18 

19 

19 

19 

20 

20 

YTAIL  27 

26 

2S  24  23 

22 

21 

20 

19 

18 

17 

16 

15 

14 

13 

12 

11 

10 

9 

0 

7 

6 

5 

DIR  7 

7 

7  7  7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

MAG  19 

5 

-0  -17  -16 

-10 

-2 

2 

0 

-3 

-5 

-i 

2 

2 

1 

-1 

-1 

1 

2 

4 

2 

-1 

1 

MAX.  GRAD  FOR  POINT  *  25  WAS  4 

AT 

19, 

8 

DIST, TOTPTS, UMCHPT  -  75S . 69  25 

DIST  *  42.0  UMCHPT-  7  OUT  OF 

WEIGHT  -  13196 


25  TOTAL  POINTS 


Figure  5.8  Results  of  attempted  verification  of  non-existent 
lineal  from  point  Q  to  point  T. 
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1  •  '*  Discussion  and  conclusions  on  verification 

The  work  on  verification  should  be  regarded  as  pilot  work  and 
too  weak  to  support  firm  conclusions.  The  question  of  evaluation  appears 
to  be  difficult  at  two  distinct  levels.  First  of  all,  how  do  we  actually 
score  the  degree  of  match  between  a  stored  lineal  track  and  a  set  of 
observed  points  from  the  image?  How  should  the  extracted  score  be 
interpreted?  Ultimately  the  answer  to  this  question  must  come  from  an 
actual  purposed  application.  The  current  experiments  reported  only 
the  number  of  points  matched  and  the  popular  RMS  distance  between 
observed  and  predicted  points.  At  the  second  level,  how  is  the  performance 
of  a  verifier  to  be  evaluated  over  a  set  of  different  lineals?  Clearly 
the  answer  is  dependent  on  the  scoring  at  the  first  level.  How  well 
did  the  verifier  perform  on  river  I  and  how  well  did  it  perform  on 
the  7  images  of  France? 

M  research  towards  solving  these  questions  needs  to 
be  done.  At  present  only  subjective  or  qualitative  statements  can 

be  made  about  the  past  work.  In  general  it  can  be  said  that  the  verifi¬ 
cation  procedure  performed  consistently  well  (i.e.  produced  good  numbers 
with  little  variance)  on  features  which  were  distinct  to  the  human 
observer.  These  were  major  roads  and  clear  field  boundaries  where 
gradient  values  were  consistently  above  threshold  and  agreed  in  location 
with  the  human  digitization.  Thus  verification  of  a  set  of  such  known 
features  appears  to  be  a  viable  means  of  confirming  a  hypothetical 
registration  transformation  obtained  from  other  evidence. 

Some  problems  arose  in  verifying  weak  features  —  features  which 
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could  be  made  out  by  a  human  using  global  tonal  context  and  perhaps 
semantics.  These  were  driveways  and  rough  field  boundaries.  Sometimes 
there  were  competing  edges  nearby,  such  as  caused  by  buildings  along  a 
road.  As  a  result,  the  gradient  values  of  feature  points  selected 
by  a  human  were  often  as  low  or  lower  than  either  the  noise  threshold 
or  the  gradient  value  on  a  competing  edge.  Lowering  the  threshold 
cannot  be  done  by  itself  because  the  large  tolerance  in  predicted 
location  would  provide  many  possible  spurious  gradient  peaks  along  the 
profiles.  A  probable  remedy  is  to  use  2-D  trend  (shape)  from  the  stored 
feature  in  order  to  better  select  the  "correct"  below  threshold  peak. 

This  should  be  tried  in  future  experiments. 

The  philosophy  and  technique  of  servoeing  deserves  further 
consideration.  Servoeing  was  installed  because  accurate  location  predic¬ 
tions  were  impossible  due  to  small  errors  in  locating  the  image  origin 
and  approximation  error  in  the  transform  T^-1.  The  belief  is  that  local 
features  can  be  used  to  achieve  better  matching  than  is  possible  using 
only  a  global  alignment.  (This  is  comparable  to  getting  accurate 
targeting  by  combining  inertial  guidance  with  image  correlation  in  the 
target  area.)  However,  servoeing  can  confuse  real  differences  in  the 
feature  with  differences  due  to  locational  error. 

The  current  servoeing  scheme  records  the  differences  (Ax^.Ay^) 

between  the  locations  (x  ,y  )  predicted  by  T  ^  applied  to  the  stored 

P  9  1  P  9  ^ 

feature  track  and  the  locations  (xq  ^,yQ  ^)  observed  by  doing  the  profile 
peak  detection  previously  discussed.  A  window  of  up  to  5  of  the  most 
recent  differences  is  saved  and  used  to  adjust  the  next  prediction  as 
follows. 
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(1) 


(2) 

(3) 


Step  1  weights  all  differences  in  the  window  equally,  a  policy 
which  might  be  changed  later.  Step  2,  however,  presents  the  most  opportunity 
for  improvement.  Perhaps  the  peak  detection  scheme  should  also  consider 
the  trend  of  the  points  in  the  window.  In  this  manner  the  tracking  should 
be  less  likely  to  go  astray  from  the  path  with  best  global  shape.  This  should 
be  tried  in  future  experiments. 

In  the  future  more  attention  will  have  to  be  paid  to  the  interpre¬ 
tation  of  the  differences  between  the  predicted  and  observed  curve  points. 
Accuracy  analysis  must  be  done  in  order  to  get  an  error  value  to  be  used 
in  interpreting  the  RMS  value  gotten  from  the  verifier.  Also,  the  patterns 
of  missing  points  should  be  scrutinized  to  detect  structured  changes 
between  the  current  imagery  and  the  mapped  imagery. 


The  next  prediction  is  made  by  transforming  the  next  point 
on  the  stored  curve  and  then  adjusting  it  according  to 
the  window  of  differences. 

N 

(x  ,y  )  =  T  (u  ,v  )  +  1/N  I  (Ax., Ay  ) 
p,np,n  ot  i  nn  l  i 

where  N  £  5. 

The  next  observed  curve  point  (x 

by  performing  peak  detection  on  the  gradient  profile 


,y  )  is  obtained 
o,n  o,n 


perpendicular  to  the  curve  and  through  point  (x  ,y  ) . 

p,n  p,n 

The  difference  between  the  prediction  and  observation  is 

recorded  as  the  most  recent  difference  in  the  window 

(queue)  and  the  least  recent  difference  is  purged. 

(A  x  ,a  y  )  =  (x  ,y  )  -  (x  ,y  ) 
n  n  o  ,n  o^n  p^n  p^n 
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LNK’s  registration  procedure  appears  very  promising.  The  procedure  will 
provide  the  full  RS&T  transformation  without  an  initial  approximate  transform¬ 
ation.  The  procedure  seems  to  be  adaptable  to  3-D  imagery  which  can  be  model¬ 
led.  LNK's  procedure  can  work  with  a  wide  range  of  features  and  is  thus  widely 
applicable.  In  particular,  abstract  vectors  formed  between  point  features  show 
promise  in  being  the  most  useful  in  rapidly  providing  an  accurate  transforma¬ 
tion. 

Since  image  to  GDB  registration  did  succeed  in  many  cases  with  a  very 
weak  representation  of  the  image,  feature  extraction  can  be  less  reliable  than 
what  is  being  attempted  by  most  A. I.  workers.  Results  reported  here  strongly 
support  the  potential  for  map-guided  image  analysis.  Future  testing  is  required 
to  diversify  the  imagery  handled,  particularly  to  try  images  with  little  or  no 
man-made  structures.  Accordingly  other  features  will  have  to  be  detected  and 
extending  the  Hough  detector  to  handle  more  complex  shapes  is  recommended.  It 
is  recommended  that  use  of  region  features  be  added  and  pass  points  be  identified 
along  arbitrary  boundary  curves,  such  as  high  curvature  points. 

The  intersection  classification  procedure  worked  very  well  on  a  limited  set 
of  non-aerial  imagery.  It  is  recommended  the  procedure  be  tested  on  a  variety 
of  imagery  in  order  to  determine  its  reliability.  In  addition  the  routines 
should  be  optimized  to  meet  time  and  space  requirements  and  should  be  extended 
to  include  intersections  of  curved  lines. 

Models  for  cartographic  classes  provide  a  means  for  disambiguating  carto¬ 
graphic  overlays.  The  disjoint  principal  components  model  was  investigated  as 
a  means  for  class  modelling.  This  method  was  selected  both  because  it  provides 
a  natural  measure  of  the  strength  of  membership  of  a  sample  in  each  class  model¬ 
led,  and  because  it  has  proven  useful  in  other  fields  such  as  chemistry. 


Some  simulation  studies  were  performed  on  synthetic  data  to  allow  for 
comparison  of  the  full  ranking  of  class  labels  as  provided  by  the  disjoint 
principal  components  model,  the  pooled  covariance  model  and  the  true  der.rir" 
functions.  This  comparison  would  not  have  been  possible  with  real  data  unless 
the  underlying  class  distributions  were  known.  While  the  disjoint  principal 
components  model  performed  poorly  in  the  simulations,  the  results  may  not  oe 
indicative  of  their  usefulness  when  using  cartographic  data.  In  fact,  the 
classes  were  easily  separated  on  the  limited  cartographic  data  available. 

Therefore,  we  recommend  extensive  testing  of  this  procedure  on  real  data 
to  provide  a  basis  for  evaluation.  In  addition,  although  the  disjoint  princip.* . 
components  model  may  not  be  suitable  for  cartographic  domains,  the  idea  of  model¬ 
ling  each  class  separately  should  be  explored  further.  In  particular,  methods 
such  as  clustering  coupled  with  measures  of  the  distance  from  a  sample  to  a 
cluster  could  be  used  to  model  classes. 

Various  disambiguation  procedures  may  be  applied  to  provide  a  unique  label 
for  each  region  from  a  set  of  possible  labels.  A  graph  theoretical  region 
matching  procedure,  the  region  adjacency  graph,  and  a  symbolic  region  matching 
method  were  investigated.  Initial  encouraging  results  on  limited  synthetic  data 
indicate  that  these  procedures  should  be  explored  further  on  more  realistic  prob¬ 
lems. 

Using  servoeing  techniques,  the  verification  of  lineals  was  successfully 
performed  in  a  few  images.  However,  much  more  diverse  testing  is  recommended 
to  assess  the  true  potential  of  the  technique.  Boundaries  of  regions  such  as 
land/water  and  forest/field  should  be  tried. 

It  is  recommended  that  methods  be  devised  for  verifying  region  features  in 
the  image  and  that  knowledge  of  elevation  be  included  in  order  to  augment  pure 
grey-scale  information.  As  much  more  needs  to  be  learned  about  interpretation 
of  the  verification  results  with  respect  to  change  detection.  It  is  recommended 
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that  individual  problems  be  studied  and  perhaps  feature  specific  decision  rules 
tried.  Verification  of  image  region  labels  is  a  complex  problem  involving  the 
weighing  of  evidence  from  several  sources  such  as  texture  measures,  intersection 
type  descriptions,  region  shape  descriptions,  location,  and  neighborhood  con¬ 
text.  We  recommend  a  study  of  the  feasibility  of  a  knowledge  based  interative 
system  for  verification.  This  study  would  entail  detailed  analysis  of  con¬ 
straints  on  image  regions  and  methods  for  structuring  these  constraints  for  in¬ 
ference. 
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Appendix  A  Sof twa’:e  Overview 

The  basic  software  used  to  support  the  research  documented  in 
this  report  was  developed  over  a  3  year  period  on  a  Univac  1108  computer  and 
then  converted  to  run  on  a  Hewlett-Packard  minicomputer.  There  were 
substantial  changes  that  had  to  be  made  due  to  word  size  differences, 
memory  size  differences,  compiler  differences,  and  the  exciting  change  from 
access  to  pre-scanned  digital  pictures  to  on-line  picture  sampling.  Nearly 
all  driver  programs  were  entirely  rewritten  and  some  enhancements  were 
made  to  subroutines. 

L.N.K.  software  currently  in  place  at  the  E.T.L.  ROSA  Lab  can 
be  subdivided  irjto  4  major  subsystems  which  are  detailed  in  appendices 
B,C,D,  and  E.  These  subsystems  are  as  follows. 

A. 1  Research  Image  Processing  System  (RIPS  or  RPS) 

RIPS  consists  of  routines  which  operate  on  grey  scale  imagery 
or  gradient  imagery.  Their  main  objective  is  to  extract  edge  features 
from  gradient  imagery.  The  most  centrally  used  routine  is  the  Hough 
straight  edge  detector.  There  are  also  routines  for  visual  display  and 
histogramming.  RIPS  data  structures  reside  in  a  set  of  6  named  common 
blocks. 

/RPSAC/  contains  the  accumulator  array  and  auxiliary  information  for 
performing  the  Hough  transform. 

/RPSCC/  contains  definitions  of  codes  for  linking  a  point  with  another 
point  in  its  neighborhood  thus  forming  generalized  Freeman 
chains. 
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/RPSGC/  contains  the  gradient  image  and  auxiliary  information  such  as 
the  gradient  histogram  and  mask  definitions. 

/RPSIC/  contains  the  last  sampled  32x32  image  and  auxiliary  information. 

/RPSSC/  contains  RIPS  system  parameters  and  other  globally  used  information 

such  as  intermediate  I/O  flags,  current  sampling  window  definition, 
and  the  DCB  and  buffer  for  I/O  with  the  edge  element  file  EEFILE. 
/RPSWC/  contains  working  space  for  a  list  of  selected  image  points  for 
use  by  the  feature  detectors 

These  common  blocks  are  added  to  program  modules  as  macros  which 
are  given  in  appendix  B  to  document  in  detail  the  common  block  content  and 
the  routines  that  operate  on  them. 

A. 2  Edge  extraction  routines  EDGEX  and  EDGEY 

EDGEX  and  EDGEY  are  two  programs  that  extract  lineal  features 
from  on-line  imagery.  EDGEX  extracts  straight  and  curved  edge  elements 
from  single  32x32  windows.  Window  settings  are  gotten  from  a  patterned 
scan  of  the  image  —  usually  a  raster  scan  of  butting  windows  is  used  to 
cover  the  image  but  more  flexible  scanning  is  possible.  The  smart 
routines  called  are  LHOUGH  and  LKTRKR  which  extract  straight  and  curved 
edges  respectively.  Records  of  detections  are  written  out  to  file 
EEFILE  for  plotting,  registration,  and/or  input  to  EDGEY. 

EDGEY  takes  the  primitive  detections  made  by  EDGEX  as  input 
from  file  EEDGEX  and  attempts  to  extend  straight  edge  elements  and  detect 
intersections  along  them.  The  gradient  masking  techniques  are  heavily 
used  by  EDGEY  to  enhance  detection  of  desired  edges  or  to  suppress 
competition  from  known  edges.  Records  of  extended  edges  or  detected 
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intersections  between  edges  are  written  out  to  file  EEFILE.  Both  EDGEX 
and  EDGEY  require  about  15  seconds  of  processing  time  per  window  sampled 
due  to  gradient  computation  time.  More  details  appear  in  appendix  C. 

A. 3  Registration  software  REG. 

The  registration  software  contains  no  image  processing  operators. 
Input  to  REG  consists  of  image  and  map  abstractions  which  are  currently 
just  a  set  of  vectors.  Output  from  REG  consists  of  a  listing  of  possible 
registration  transformations  (rotation  and  translation)  along  with  an 
evaluation  of  their  merit.  At  the  heart  of  REG  is  a  set  of  routines  that 
perform  a  clustering  operation  in  3-space.  The  clustering  is  done  by 
binning  points  into  a  set  of  10x10x10  bins.  2,3,  or  4  iterations  of 
binning  are  typically  required  to  get  the  bin  size  down  to  the  size 
of  error  allowed  in  the  final  transformation  parameters.  Appendix  D 
has  details. 
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A. 4  Verification  software  VERIFI 

A  fourth  program  and  auxiliary  routines  exist  for  top-down 
verification  of  lineals  in  an  image.  Lineals  are  known  a  priori  from  a 
map  or  from  previous  image  analysis  and  are  to  be  referred  to  raw  imagery 
via  the  registration  transform  for  verification.  The  software  transforms 
each  point  along  the  lineal  using  the  registration  transform  and  then 
performs  a  search  for  a  match  to  the  point  by  examining  gradient  profiles 
perpendicular  to  the  directionOf  the  curve.  A  match  measure  is  built  up 
for  the  entire  lineal  by  averaging  the  squared  distance  between  the  predicted 
and  observed  locations  of  individual  points.  A  special  feature  of 
VERIFI  is  its  ability  to  make  local  adjustments  (servo)  to  the  predicted 
location  of  points  based  on  past  observations.  This  allows  for 
legitimate  departure  from  strictly  linear  registration  transformations  due 
to  image  deformations,  relief  displacement  or  actual  movement  of  the  feature 
over  time.  Details  of  the  verification  software  are  in  Appendix  E. 
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Appendix  B:  RIPS  Software 

The  Research  Image  Processing  System  (RIPS  or  RPS)  is  composed 
of  routines  that  operate  on  digital  images  for  the  purpose  of  feature 
extraction  or  display.  While  there  are  driver  programs  which  organize 
these  routines  for  interactive  exploration  of  imagery,  the  primary  function 
of  RPS  software  is  as  a  subroutine  library  for  image  processing  programs 
such  as  those  documented  in  Appendices  B,C,  and  E. 


Routine  Arguments 


Function 


ARRAY  (IDELX.IDELY, 

IGXLO, IGYLO) 

array 

sampling 


CNVTPR  (THETA, R.NCOLS, 

NROWS ,NPT  S  ,X1 ,Y1, 
polar  to  X2,Y2) 

rectangular 
conversion 


FILTR  (AX , AY , BS , BY , CX , CY 

TYPE) 

edge 

masking 


This  routine  moves  the  film  stage 
to  (IDELX.IDELY)  relative  to  the  current 
position  and  then  obtains  the  32x32 
sample  wirdow.  (IGXLO, IGYLO)  is 
returned  as  the  updated  global  stage 
position.  ARRAY  also  updates  the  image 
common  block  with  details  about  the 
newly  taken  sample. 

CNVTPR  converts  a  Hough  detection  from 
polar  form  (THETA, R)  to  a  directed  edge 
from  point  (X1,Y1)  to  point  (X2,Y2)  which 
lie  on  the  boundary  of  the  sampling 
window.  The  window  is  defined  as  NROWSx 
NCOLS  which  is  typically  32x32  for  the 
ROSA  lab.  The  darker  side  of  the  edge 
lies  toward  the  right  when  traversing 
from  (X1,Y1)  to(X2,Y2).  NPTS  is  returned 
as  2  iff  conversion  is  successful. 

Used  to  filter  or  mask  out  an  existing 
edge  (AX, AY)  to  (BX,BY)  from  the  current 
window.  (CX,CY)  is  the  current  array 
center  in  global  stage  coordinates.  TYPE 
specifies  what  kind  of  masking  is  to  be 
done  by  the  next  gradient  operators  to 
be  applied  as  follows. 


k 
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Routine 


Arguments 


Function 


FIXANG  (RESLTN) 

accumulator 

array 

setting 


GRADDL  (XLO,XHI,YLO,YHI, 

TYPE) 

gradient 
magnitude 
and  direction 


HISTOG  (TYPE) 

image  or 
gradient 
histogram 


TYPE=1  to  mask  out  all  points  with 

same  gradient  direction  as  edge 
TYPE=2  to  mask  out  all  points  with 
same  direction  or  180°  + 
same  direction  as  edge 
TYPE=3  all  of  the  above  +  mask  out  all 
points  along  the  edge  (AX, AY) 
(BX,BY)  regardless  of  gradient 
direction. 

Masks  are  actually  set  in  gradient  image 
common  by  calling  routine  MASKS. 

This  routine  sets  up  the  accumulator 
array  in  preparation  for  performing 
the  Hough  transform.  RESLTN=  8 ,16, 'or 
32  depending  on  what  angular  resolution 
is  being  used  for  gradient  direction 
calculations.  The  RIPS  accumulator 
common  block  is  intialized  Tor  use  by 
routine  LHOUGH . 

Computes  a  32  directional  gradient  at 
each  point  within  the  subwindow 
X  G  (XLO,XHI)  Y  G  (YLO.YHl) .  Iff  TYPE=0 
then  thinning  Is  also  performed  on  the 
resulting  gradient  image.  The  resulting 
gradient  image  is  stored  in  gradient 
image  common . 

Computes  histogram  of  image  grey  scales 
(if  TYPE=”IM”)  or  gradient  image  (if 
TYPE="GR") .  The  histogram  is  stored  in 
image  common  or  gradient  image  common 
fo*  use  by  SELECT  or  may  be  optionally 
printed. 


L FOCUS  (NHITS , ITHRES , IRESL) 

focus 

Hough 

detection 


Used  to  focus  Hough  detections  made  with 
a  coarse  11°  angular  resolution  to  a 
2°  angular  resolution.  The  list  of 
NHITS £5  coarse  detections  is  passed  to 
LFOCUS  from  LSCAN  through  accumulator 
common.  ITHRES  is  the  threshold  for  the 
original  detection.  If  the  strength  of 
the  fine  detection  falls  below  2/3  of 
ITHRES  the  detection  is  discarded.  IRESL 
is  the  gradient  direction  resolution  which 
is  8,16,  or  32.  Detections  are  written  to 
file  EEFILE. 
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Routine 


Arguments 


Function 


LHOUGH 

Hough  line 
detector 


LKTRKR 

continuous 

curve 

detector 


LSCAN 

accumulator 
array  peak 
detector 

MASKED 

point 

masking 


MASKS 

mask 

setting 


(XCENTR.YCENTR) 


(XKO ,XHI , YLO.YHI , 
LTHRSH) 


(NTHRES , NDET) 


(X.Y.D) 


(IOP , I , J,K,L) 


Computes  the  Hough  transform  for  snaig... 
edge  detection.  Details  of  the  detector 
are  assumed  to  be  set  up  in  accumulator 
common  by  FIXANG.  The  input  points  are 
taken  from  a  list  in  work  common.  Output 
is  the  accumulator  array.  (XCENTR.YCENTR) 
define  the  origin  of  the  coordinate  sys¬ 
tem  to  be  used  and  is  typically  the 
window  center  (16,16).  Detection  of 
peaks  in  the  accumulator  array  is  left 
for  LSCAN. 

Detects  sets  of  high  gradient  points 
organized  into  continuous  curves.  Points 
have  been  linked  by  WRKLNK  in  the 
gradient  image.  (XL0.XH1.YL0.YHI)  define 
the  search  window  and  LTHRSH  defines 
the  minimum  number  of  points  which  make 
up  a  curve.  Detected  curves  are  chain- 
encoded  and  written  out  to  file  EEFILE. 

Detects  local  maxims  in  the  accumulator 
array  which  exceed  threshold  NTHRES .  Up 
to  5  detections  are  reported  in  accumulalo 
common  and  the  number  of  detections  is 
returned  in  NDET. 

Logical  function  returns  true  iff  point 
(X.Y)  with  gradient  direction  D  is  to 
be  masked  out  of  consideration  due  to 
location  (X.Y)  or  gradient  direction  D. 

A  previous  call  to  FILTR  or  MASKS  must 
have  been  issued  to  define  the  masking. 

Used  to  build  masks  for  location  or 
gradient  direction  masking  as  follows. 
I0P=1  clears  all  masking 
I0P=2  mask  off  (X.Y)  if  to  the  right 
of  halfplane  defined  by  (I,J)- 
(K.L) 

I0P=3  mask  off  direction  D  if  D  6 

(I,J) 

I0P=4  print  out  current  mask 
definitions. 


196 


Rout ine 


Arguments 


Function 


MLTDIR  (X,Y , RES ,MAG, 

DIR.IERR) 

multiple 

resolution 

gradient 


PRINTA  (NUNIT) 

point  accum¬ 
ulator  array 

PRINT I  (XLO,XHI,YLO,YHI, 

CHAR) 

print 

grey 

scales 

SELECT  (XLO, SHI ,YLO ,YHI , 

THRSHL) 

select 

heavy 

gradient 

points 


Computes  directional  gradient  at  point 
(X,Y)  using  directional  resolution  of 
RES»8,16,  or  32.  Magnitude  MAG  and 
direction  DIR  of  the  gradient  are 
returned.  IERR^O  if  error  encountered. 
(X,Y)  must  be  2,4,  or  5  pixels  off 
window  boundary  for  computation  of  the 
8,16,  or  32  directional  gradients 
respectively. 

Prints  out  the  contents  of  the  accumulator 
array  on  I/O  unit  NUNIT.  A  utility 
routine  for  human  useable  output  only. 

Utility  routine  to  print  out  human 
useable  grey  scale  plots  of  the  image 
(CHAR="IM")  or  gradient  image  (CHAR="GR") 
of  subwindow  defined  by  X  6  (XLO,XHI) , 

Y  6  (YLO.YHI). 

Routine  for  automatic  selection  of  a 
threshold  THRSHL  such  that  a  fraction 
FRACT  of  the  gradient  values  exceed 
THRSHL.  HISTOG  is  called  to  get 
a  histogram  and  then  THRSHL  is  set  from 
the  histogram  and  FRACT.  Points  that 
exceed  the  threshold  are  then  placed 
in  work  common  for  the  detection  routines. 

Points  are  selected  from  subwindow 
defined  by  X  G  (XLO,XHI),  Y  6  (YLO,YHI) 


WRKLNK  (NCIRCL.SRCANG, 

DRTHRS) 

link  curve 
points  to 
neighbors 


Finds  best  connecting  point  for  each 
point  in  work  common  in  both  the  forward 
and  backward  direction.  Connections 
are  used  later  by  l.KTRKR  to  extract 
long  continuous  elm  Ins  Indicating  curved 
edges.  NCIHCl.  defined  sl'.e  of  neigh¬ 
borhood  searched  lor  connecting  point. 
Connecting  point  must  lie  within  the 
angular  range  defined  by  SRCANG  and  have 
gradient  direction  within  DRTHRS  of 
the  first  point.  Thus  SRCANG  and  DRTHRS 
are  used  to  control  allowed  curvature. 
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i 

i 


r* 

r 


r 

c; 


C 


c 

r 

c: 


c: 

c 


c 

c 

c 

c: 

c 

r 

c 


r 


c 

r 

c 


**##RIPS  ACCUMUI  ATOR  ARRAY  H.P.  VERSION  29DF.C70******* 

TNI  l  UFR  Kl  0  RHI  .RIUMP  ,  ATT  .NR  ,N1  META  ,  TWTl)  I  H 
INTEGER  DGRFLG , I OPTR . HTPTR .EININO 
REAL  COSINE ,SINE , KTHFTA 
INTEGER  RADIUS .ANGLE, WEIGHT. WGDUMY 


COMMON  AREA  IS  1664  16-BIT  WORDS  LONG  .  OR  13  i?B  WORD  BLOCKS 

ICOMMON  /RPSAC/  I'LENACC  .WGDUMY  .RLO.RHI  .R.TUMP  ,  TWIDTH  ,  NR  .NTHETA, 

+  DGRFLft  ,  LSCNIJM  ,  RADI  US  ( 5 )  ,  ANGI..E  ( S )  .WEIGHT  (5)  , LOP TP (3?)  . 

+  HIPTR(3R) ,FININ0(32> , 

+  COSINE. (64)  .SINE  (64)  . KTHETA ( 64  )  ,  ACC (32. 33) 


ACC 

KTHFTA 

FININO 

DGRFLG 

THETA 

RLO.RHI  ,  R JUMP 
NR 

NTHETA 

COSINE 

SINE 

TWIDTH 

WGTHOU 

LOPTR . HIPTR 

RADIUS .ANGLE, WEIG 
LSCNUM 


DIRECT  T'-'* 
(  ) 


ACCUMULATOR  ARRAY  FOR  THETA-RADIUS  SPACF 
VALUES  FOP  POSSIBLE  ANGLES  IN  DEGREES 
FOR  TRANSLATION  BETWEEN  ANGLE  INDEX  AND  EINI. 

INDICATES  IF  ANGLES  IN  DEGREES  (  )  OR  FINIS 

VALUE  OF  ANGLE  SET  IN  REAL  RADIANS 
DO-LOOP  DEFINITION  OF  RADIUS  VALUES 
NUMBER  OF  RADIUS  VALUES 

NUMBER  OF  ANGLES  USED  AND  STORED  IN  KTHETA  ARRAY 

COSINES  OF  ANGLES  IN  KTHETA  ARRAY 

SINES 

WIDTH  OF  STRAIGHT  LINE  TEMPLATE 

INDICATES  WEIGHTED  (=i)  POINT  OR  WEIGHT  i  POINT  (NOT  i 
POINTERS  TO  KTHETA  ARRAY  TO  FILTER  TRANSFORM  ACC.  TO 
GRADIENT  DIRECTION 

HT  PEAK  RESPONSES  FOUND  IN  ACCUMULATOR  ARRAY 
NUMBER  OF  PEAK  RESPONSES  " 


C*****END  ACCUM  PROC  H.P.  VERSION  29DEC78  R . I . P , S . ***** 


> 


L  * 

C  IMAGE  PROC  H.P.  VERSION  31  OCT  7B  ***R TPS*** 

C  ■» 

C  STORAGE  AND  FORMAT  FOR  COMMON  BLOCK  CONTAINING  IMAGE 
C 

INTEGFR  LENIMC  .  IMNAM , NC0DE8 , NR0W8 ,NCOI  S .LEVELS , I MAG , NCOI  WD 
INTEGER  HI5TIM  IMGP AD . VXLO , VXHI .VYLO  .VYHI , HISTFG 
[COMMON  /RPSIC/  ILENIMC  ,  IMNAM<  3 )  .NROWS  .NCOLS  .NCODFS  , NCCJt  WD  . 

4  LEVELS, VXLO, VXHI , VYLO, VYHI ,HISTIM(64) , 

4  IMGP  AD  ,  I  MAG  ( 32  ,  32 ) 


LENIMC 

r: 

C  IMNAM 
C  NROWS. NCOLS 
C  LEVELS 
C  T  MAG 

C  NCOLWD 
C  HISTFG 
T  IIT^TTM 
r  IMGP AD 
C  S  .  .  . 

C  U  ,  ,  , 

c 


NUMBER  OF  WORDS  IN  COMMON  (0  IF  NOT  INITIALIZED) 
(ELSE  LENIMC  =  1100) 

ASCII  NAME  OF  IMAGE. 

NOMBNER  OF  ROWS  ANI)  COLUMNS  OF  IMAGF 

NUMBER  OF  GREY  LEVELS  USED  (  0  , 1  I.  EVELS-i  ) 

IMAGE  ARRAY 

ARRAY  MAPPING  CONSTANT  OF  OLD  UNIVAC  RIPS 
HISTOGRAM  VALIDITY  FLAG  (NOT  USED  30MAR79) 

HISTOGRAM  ARRAY 

PADDING  FOR  PROPER  EXEC  CALL  TO  SAMPLE  ARRAY 
SCREEN  WINDOW  DEFINITION 

VIRTUAL.  WINDOW  DEFINITION  (IN  ABSOLUTE  FILM  COORDS) 


C  END  IMAGE  PROC 


#*#RIPS*¥* 


r: 

c 

r 


chsdfg  f'wnc  up.  ve r b i on  22  nov  vu  *»*ni";*** 

LINE  CHASING  AND  LINKING  CODES  FOR  GFNFRAI  FREEMAN  COWS 


INTEGER  DX , DY  ,  COMPMT 

1  COMMON  /RPSCC/'jPX ( 68 )  ,DY(68>  , COMPMT < 68) 


DX  <  LINK )  IS  X  INCREMENT  FOR  FOLLOWING  L  INK 
SIMILAR 


PX 
DY 

COMPMT  100  DEGREE  COMPLIMENT  OF  LINK 


CHSDFS  PROC  H.P.  VERSION  22  NOV  70 
FTN4.L ,T 

BLOCK  DATA  RPSCH 


#*#RIPS#** 


C 

C 

C 

C 

C 

C 

C 

c 

c 

c 

c 


INCLUDE  RPSSYC 
INCLUDE  RPSIMC 
INCLUDE  RPSGRC 
INCLUDE  RPSWKC 
INCLUDE  RPSACC 
INCLUDE  RPSCHC 

DATA  DX  /i.i.O,- 1,-1 ,-i, 0,1, 

2,2,2, i, 0,-1, -2,-2, -2, -i, 0,1, 

2, 3, 3, 3, 2,  i ,0 ,-i ,-2,-3, -3, -3, 


-i ,0  ,i  , 

2  ,3,3 ,2  ,-2,-3, -3, -2 ,-i ,0 ,  1 ,4,4,4 , 1 ,0 , 
-i  , -4  ,  -4 , -4  ,  -4 , -3 , -2  , 2 , 3 , 4 ,4 , 3 , 2 , - 2 , -3 , -4/ 

59  45  46  47  60 
50  44  34  35  36  37  61 
57  43  33  18  19  20  21  38  62 

56  32  17  6  7  8  9  22  48 

55  31  16  5  #  1  10  23  49 

54  30  15  4  3  211  24  50 

68  42  29  14  13  12  25  39  63 

67  41  28  27  26  40  64 
66  53  52  51  65 


DATA  DY  /0 ,~i ,-i  ,-i  ,0,1  ,1  ,1  , 

+  1  ,0 ,-i, -2, -2, -2,-1 ,0,1, 2,2,2, 

+  2,1 ,0 ,-i ,-2,-3, -3, -3, -2,-1 ,0 ,1 ,2  3 ,3,3, 

+  3 , 2 , -2 , -3  > -3 , -2 , 2 , 3 , 4 , 4 , 4 , 1 , 0 , -1 . -  4 , -4 , 

+  -4.-1  ,0,1 ,2, 3, 4, 4, 3, 2, -2, -3, -4,-4, -3,-2/ 

DATA  COMPMT  /5, 6, 7, 0,1, 2, 3, 4, 15, 16, 17. 18. 19,20.9.10,11, 12.1 
+  29,30,31,32,33,34,35,36,21 ,22,23,24,25,26,27,28 , 

+  41,42,43,44,37,38,39,40,51 ,52,53,54,55,56,45,46 , 

+  47,48,49,50,63,64,65,66,67,68,57,50.59,60,61 ,62/ 

C 

C  COMPMT ( 0 )  IS  UNDEFINED 

C  CHSDFS  PROC  H.P,  VERSION  22N0V7R 
END 


*##R  IPS**# 


3.14 
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GRADFS  PROC 


P.  VERSION  31  OCT  7R  ##*R  IPS*** 


FORMAT  AND  ST OR A OF  OF  COMMON  STORING  GRADIENT  IMAGE 

INTEGER  LENGRC  .GPNAM  .HISTGD  .HSXHI  ,  HSXI  .0  ,  HSYHI  ,1-ISYl  O.HEST  ' 

INTEGER  MAG , DIR  .  SUPRES , G < 32 , 32 ) , F L DIR ( 32 , 32 >  , BE  DIR ( 32 , 32 > 

INTEGER  NTRPL..S  ,  NPAIRS .MKDIRI  .MKDIP2 

REAI..  MKLOCA  ,  MKLOCB  ,  MKL.OCC 

INTEGER  RESLTN , WGTHOU 

REAL  HSMEAN.HSVAR 

I  COMMON  /RPSGC/I LENGRC ,GRNAM( 3)  , RE  SLTN  .WGTHOU  .HSMF.AN  .HSVAR  , 
HSXL  0  ,  HSXEI I  .  HSYL  0  ,  HSYHT  ,  HTSTFE  ,HISTGD(64)  . 

(*  MAXMAG, 

Hi  MAG  ( 31?  ,  3?  )  , DIR ( 32 , 3?  >  SUPRES  (32,3?)  , 

4  NTRPL.S  ,  MKLOCA <  4 )  , MKLOCB ( 4  )  ,MKI..OCC<4)  , 

*.  NPATRS,MKDIRi(4)  , MKDT  R2 ( 4 ) 

EQUIVALENCE  ( G  , MAG )  , ( FLDIR , DIR ) , (BI.DIR , SUPRES) 


LENGRC 

GRNAM 
RESLTN 
WGTHOU 
HSMEAN , HSVAR 
HS.  .  . 

HISTF'L 

HISTGD 

MAG 

MAXMAG 

DIR 

SUPRES 
NTRPI..  S 
MKLOCA/B/C 
NPAIRS 
MKDIRI /2 
G 

FLDIR  ,  BI..D  IP 


NUMBER  OF  WORDS  IN  COM  'ON  <0  IF  NOT  INITIAL  I /ED'' 
(ELSE  LENGRC  »  31S2  ) 

NAME  OF  IMAGE  FROM  WHICH  GRADIENT  EXTRACTED 
RESOLUTION  OF  GRADIENT  DIRECTION  (8 ,16, OP  32) 
PARAMETER  FOR  LHOLIGH  TRANSFORMATION 
MEAN  AND  VAR  OF  NOISE  IN  GRADIENT 
WINDOW  ON  GRADIENT  IMAGE  USED  FOR  HISTOGRAM 
HISTOGRAM  VALIDITY  FLAG  (NOT  USED  3PMAR7y ) 
HISTGDRAM  OF  GRADIENT  IMAGE 
GRADIENT  MAGNITUDE  ARRAY 

MAXIMUM  GRADIENT  VAL.UF  +  1  =  #GRAI)IFNT  LFVELS 

GRADIENT  DIRECTION 

SUPPRESSION  FLAG  USED  IN  THINNING 

NUMBER  OF  HALFPLANE -DEFINING  TRIPLES 

A  ,  B  , C  COEFFICIENTS  DEFINING  HALFPI  ANF  MASK 

NUMBER  OF  PAIRS  DEFINING  GRADIENT  DIRECTION  MASK 

DIRECTION  RANGE  DEFINING  GRADIENT  DIRECTION  MASK 

ALIAS  FOR  GRADIENT  MAG 

FORWARD  AND  BACKWARD  EDGE  DIRECTION 


END  GRAD IT  PROC 


***RIPS*** 
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I 


V. 

c 


c 

c 


c 

i: 

c 

t: 

c 

c 

c 

c 

c 

c 

c 

c; 

c 

c 

c 

c 

c 

c 

c 

r 


SYSTEM  PROC  HP,  VERSION  310CT78  #*#R  I  PS*** 


MODES  ,1'ARAMf  ILRS  DEBUG  FI  AGS  FOR  RIPS 

INTFRFR  I  NON  I  1  ,  0  1  ON  I  I  ,  DAUNT  T  ,  1  MUNI  T 
INTEGER  1DODFF  ,  FE..NAMF  .FFPOFR 

INTEGER  R  IP  MOD  ,  DEBUG  ,  L  ENSYC , PWIDTH  SY  I  NAM  ,  SYNCH!  .SYNKUW 
I  NTEGER  SXI.O  ,SXHT  .SYL  0  ,SYHT  ,  SYVXLO  ,  SYVXIII  ,  SYVYI  O  .SYVYI  1 1 
'  1COMMON  /RPSGCt/l  I.  ENSYC  ,  INUNIT  .OTUNIT  .DBUNIT  ,  TMUNIT  , 

4  PWIDTH  ,  R  IP  MOD  ,  SYINAM  (  3  )  .SYNROW  SYNCOI  , 

4  SXLO.SXHI  ,  SYI..Q  ,  SYHI  ,  SYVXLO , SYVXHT  , 

4  SYVYL.O , SYVYHI , DEBUG ( 20 )  , FFNAMF <  3  >  > IBCBFF ( 144 )  , 

4  FEPUFR(AA) 

L. ENSYC  LENGTH  IN  WORDS  OF  COMMON  (0  IF  NOT  INITIALIZED) 

(  ELSE  I  FNSYS  -  187  ) 

SYINAM  NAME  OF  CURRENT  IMAGE 

SYNCOL.  NUMBER  OF  COLUMNS  OF  CURRENT  WINDOW 

SYNROW  NUMBER  OF  ROWS  OF  CURRENT  WINDOW 

INUNIT  INPUT  UNIT  NUMBER  ,  TYP ICAL  I.  Y  FOR  UNIT  S  INPUT 

0 TUN IT  OUTPUT  UNIT , TYPICALLY  PRINT  UNIT  b 

TMUNIT  INTERACTIVE  TERMINAL  UNIT  NUMBER ,  I.E.  1  OR  4 

RIPMOD  RIPS  MODE,  I.E.  BATCH=0  OR  INTERACTIVE-! 

DEBUG  ARRAY  OF  DEBUG,  OR  INTERMEDI ATE  I/O  FLAGS,  ONE  OR 

MORE  FOR  EACH  RIPS  MODULE 

S...  SCREEN  WINDOW  ON  IMAGE  IN  LOCAL  COORDINATES 

SYV .  .  .  VIRTUAL  WINDOW  OF  IMAGE  IN  ABSOLUTE  FILM  COORDINATES 

P WIDTH  WIDTH  OF  PRINT  LINE  IN  CHARACTERS,  ^7?  OR  13? 

FENAME  NAME.  OF  EDGE  ELEMENT  FILE,  SHOULD  BF  "EEFILE " 

IDCBEE  DATA  CONTROL.  BLOCK  FOR  EEFILE , OPENED  AND  CLOSED  BY  DRIVER 

EEBUFR  BUFFER  FOR  OUTPUT  OF  DETECTED  EDGE  ELEMENTS  TO  EEFILE 

END  SYSTEM  PROC  ***RIPS*** 


C 

C 

r 

C 


c 

c 

c 


c 

r 

c 

r: 

c 

c 

c 

c 


WORKCM  PROC  H.P.  VERSION  310CT78  ***RIPS*** 

DEFINES  WORK  COMMON  BLOCK  WHERE  SETS  OF  SELECTED  POINTS  STORED 


INTEGER  LENWKC  ,  I..ENWRK  ,  WX<  128  )  ,WY(i?8)  ,WGRDIR(i?8) 
INTEGER  UGR(128)  , WFL.DIR  (IPS)  ,WKNAM(3) 

EQUIVALENCE  ( WFL.DIR  ,  WGRDIR  ) 

J COMMON  /RPSWC/J  l  ENWKC , LENWRK , WKNAM , WX , WY , WGR , WGRDIR 


LENWKC 

WKNAM 
LENWRK 
WX  ,  WY 

WFI.DIR  , WBL.DIR 

WGRDIR 

WGR 


TOTAL  LENGTH  IN  WORDS  OF  COMMON  ( 0  IF  NOT  TNTTTTAI  T 7 E D  > 
(  ELSE  LENWKC  =  517  ) 

ASCII  NAME  OF  PICTURE  FROM  WHICH  POINTS  SELECTED 

NUMBER  OF  POINTS  SELECTED 

X.Y  COORDINATES  OF  SELECTED  POINTS 

FORWARD  AND  BACKWARD  EDGE  DIRECTION  AT  POINT 

GRADIENT  DIRECTION  AT  POINT 

GRADIENT  MAGNITUDE  AT  POINT 


END  WORKCM  PROC 


***RIPS*** 


l; 
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Appendix  C:  Edge  extraction  routines  EDGEX  and  EDGEY. 

Two  Batch  programs  exist  which  extract  lineal  edge  data  from 
on-line  imagery.  EDGEX  performs  a  patterned  scan  of  the  image  extracting 
straight  and  curved  edge  elements  from  sampled  windows.  One  record  is 
written  to  file  EEFILE  for  each  edge  element  extracted.  The  scan  pattern 
is  defined  by  giving  a  window  spacing  in  stagels,  the  beginning  window 
center  in  stagels,  and  the  trajectory  of  subsequent  window  centers  in 
terms  of  a  Freeman  chain  code.  The  following  output  gives  a  sample  run 
of  EDGEX  scanning  around  a  road  intersection.  Typically  EDGEX  is  used 
by  repeating  a  raster  scan  pattern  across  the  entire  image.  EDGEX 
allows  checkpointing  after  each  patterned  scan  is  complete,  or  the  user 
can  repeat  the  scan  pattern  over  the  next  region  of  the  image. 

EDGEX  requires  about  15  seconds  per  sampled  window;  almost 
all  of  that  time  being  consumed  by  the  gradient  operator.  LHOUGH  and 
LKTRKR  are  the  smart  routines  called  and  extract  straight  or  continuously 
curved  edge  elements  respectively.  Detections  are  written  to  EEFILE 
which  must  be  saved  by  copying  if  the  checkpointing  facility  is  being 
used.  The  following  output  gives  routines  loaded  with  EDGEX  and  sample 
output  of  the  program. 

EDGEY  is  a  batch  program  which  takes  the  primitive  detections 
made  by  EDGEX  as  input  and  attempts  to  extend  straight  edge  elements  and 
detect  intersections  along  them. 

For  each  primitive  straight  edge  element  input  to  EDGEY,  the 
processing  logic  is  as  follows. 


1)  The  stage  is  positioned  so  that  the  edge  element  should  be  centered 


in  the  window.  Competing  edge  directions  are  masked  off  and  the  Hough 
detector  is  called  to  verify  the  edge.  If  the  edge  is  missed 
(happens  infrequently)  then  go  to  step  5  .  If  the  edge  is  detected 
adjust  (servo)  the  stage  to  center  the  edge  element  in  the  window. 

2)  Search  along  the  detected  edge  for  nearly  perpendicular  intersecting 
straight  edges.  When  searching,  the  existing  edge  direction  d  and 

d  +  180°  are  masked  off  to  remove  contention.  Also  the  stage  is 
toggled  so  that  almost  an  entire  window  is  visible  on  either  side 
of  the  existing  edge.  (See  Figure  2.10). 

3)  Attempt  to  extend  the  current  edge  in  the  forward  direction.  If 

a  forward  extension  exists  go  to  step  2,  else  continue  at  step  4. 

4)  Attempt  to  extend  the  current  edge  in  the  backward  direction.  If 

a  backward  extension  exists  perform  step  2  search  for  intersecting 
edges  and  repeat  this  step.  When  no  further  backward  extension  is 
detected  continue  at  step  5. 

5)  Negate  any  stage  adjustment  made  in  step  1.  Write  a  record  out  to 
EEFILE  defining  the  full  extent  of  the  edge  found.  (Any  detected 
intersections  were  written  out  to  EEFILE  when  detected.) 

The  chief  defect  in  EDGEY  as  currently  implemented  is  that  the 
routine  does  not  remember  the  edges  it  has  already  worked  on.  Quite 
often,  EDGEX  will  detect  Several  pieces  of  a  long  straight  edge. 

EDGEY  is  then  condemned  to  extend  and  search  from  each  piece,  often 
achieving  multiple  identical  results.  Theoretically  the  duplication  is 
easy  to  remove  at  a  later  stage,  but  a  great  deal  of  time  is  currently 
wasted  working  over  the  same  edge. 
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C.  1 _ Loader 

Command 

RE , %EDGEX 

RE,%ARRAY 

RE , %RPSFX 

RE,%RPSGL 

RE,%RPSHS 

RE,%RPSSE 

RE , %RPSHG 

RE,%RPSFO 

RE,%RPSLS 

RE,%RPSWR 

RE,%RPSLK 

RE,%RPSPR 

RE,%RPSPA 

RE ,%RPSML 

RE,%RPSVG 

RE , %STAGE 

RE,%RPSMK 

RE , %RPSCH 

RE , %RPSVT 

RE , %RPSAA 


commands  tor  creating  EDGEX 

Comment 
main  routine 
window  sampling  routine 
routine  initializes  accumulator  array 
computes  32-directional  gradient 
histograms  gradient 
selects  strong  gradient  points 
computes  Hough  transform 
focuses  Hough  transform 
scans  accumulators  for  peaks 
links  edge  points  to  best  neighbors 
extracts  continuous  curves 
prints  grey  scales  if  required 
prints  acc.  array  if  required 
computes  gradient  at  single  point 
converts  polar  vector  to  rectangular 
positions  stage 
sets  gradient  masks 

block  data  routine  for  chain  code  common 
converts  vector  to  gradient  direction 
worker  subroutine  for  RPSLS 
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C . 2  SAMPLE  RUN  OF  EDOEX 


>c 


jC 

* 

c 

I 

( 

u 

c 

It 

c 

«• 

c 

B 

c 


( 


c 

c 

c 

< 

c 

c 

c 

t 

< 


: RUN.EDGEX ,6 ,6 
GIVE  IMAGE  NAME ( 3A2 ) - 
4621-3 

GIVE  FRACTION  OF  POINTS  TO  SELECT < REAL/FREE >- 

o.os, 

GIVE  10  I/O  FLAGS  < INT /FREE ) - 

0,0, 0,0, 0,0, 0,0, 0,0, 

GIVE  GLOBAL  COORDINATES  OF  ORIGIN  < INT/FREE  >- 
5000,5000,  must  A KUC  ST*.]*  pOSITionttd 
•  PTS  ON  LINE  6  *  PTS  ON  CHAIN  7  (2  INT/FREEI- 

20 ,20  , 

FIRST  WINDOW  OFF  OR IGIN-DELX , DEL Y  < INT /FREE ) - 
6120,3072, 

GIVE  SAMPLING  TRAJECTORY  !  LSIZE,N 
<LINKS(I),I-1,N)  UNT/60I1) 

200,8, 

54117755 


$  Samples  I'lJicA.+ed 
»CA,r,  pa/Hcrv  . 

/it=  6+k  ?  YA. 

Ou/  n  J  o  w  s  5  A  ?  w  . 


IERR-  4  IN  OPENING  FILE  EEFILE 

IERR”  0  IN  WRITING  LABEL  ON  EEFILE 

GRADDL  ON  IMAGE-4621-3  WINDOW-1100011248  7952  8200 

GRADDLiNOW  THIN  MAXMAG , TYPE-  155  0 

SELECT i THRESHOLD-  82 

SELECT :LENWRK-  66 

OSIGNIFIGANT  LOCAL  MAXIMA  WITH  THRESHOLD  =  20  WERE  FOUND  AT 


ANGLE  OR  DIRECTION  RADIUS  WEIGHT 
*■  "i7  -3  25 

LFOCUS ; 1 1 159 ,  0207  11159,  7959  179,0,  -3,0,  WT-  15  20,32  20 ,  1 

LKTRKR :  0  CHAINS  OUTPUT  OF  MIN  LENGTH  20  -  0  LINKS  TOTAL 

<  ***VIRTUAL**¥  11000  11248  7952  8200*SCREEN*  1  32  1  32  ***** 

1234567890123456789012345678901234567890 
32  * 

*  31 

30 
29 

'  28 

27  X 
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<LINKS<I),l-i,N>  < INT/60 I 1 )  '  ~~r~ 

aao,8, 

S41177SS 

IERR-  4  IN  OPENING  FILE  EEFILE 

IERR-  0  IN  URITINC  LABEL  ON  EEFILE 

GRADDL  ON  IMAGE-4621-3  WINDOW-! 1 0001 1248  7952  8200 

GR ADDL : NOW  THIN  MAXMAG , TYPE-  155  0 

6ELECT (THRESHOLD-  82  - 

SELECT iLENURK-  66 

OSIGNIFIGANT  LOCAL  HAXIhA  WITH  THRESHOLD  «  20  WERE  FOUND  AT 

ANGLE  OR  DIRECTION  RADIUS  WEIGHT 
17  -3  25 

LFOCUS : 1 1 159 ,  8207  11159,  7959  179,0,  -3,0,  WT-  15  20,32  20,  1 

LKTRKR:  0  CHAINS  OUTPUT  OF  MIN  LENGTH  20  -  0  LINKS  TOTAL 

***VIRTUAL***  11000  11248  7952  B200*SCREEN*  1  32  1  32  ***** 

123456789  0 1 234567890123456789  0123456789  0 


GRADDL  ON  IMAGE-4621-3  UINDOU-lOBOOl 1048  7952  8200 
GRADDL. NOW  THIN  MAXMAG , TYPE-  107  0 

SELECT : THRESHOLD*  50 
SELECT iLENURK-  30 

0LSCAN-.  NO  LOCAL  MAXIMA  FOUND  WITH  THRESH  -  20 

LKTRKR!  0  CHAINS  OUTPUT  OF  MIN  LENGTH  20  -  0  LINKS  TOTAL 

GRADDL  ON  IMACE-4621-3  WINDOW-1060010848  7752  8000 
GRADDL: NOW  THIN  MAXMAG , TYPE-  28  0 

SELECT : THRESHOLD-  18 
SELECT ;LENWRK-  34 

OLSCANi  NO  LOCAL  MAXIMA  FOUND  WITH  THRESH  »  20 

LKTRKR !  0  CHAINS  OUTPUT  OF  MIN  LENGTH  20  -  0  LINKS  TOTAL 

GRADDL  ON  IMACE-4621-3  WINDOW-1 08001 1 048  7752  8000 
GRADDL: NOW  THIN  MAXMAG, TYPE-  8S  0 
SELECT: THRESHOLD-  11 
SELECT iLENURK-  88 

OSIGNIFIGANT  LOCAL  MAXIMA  WITH  THRESHOLD  >  20  WERE  FOUND  AT 

ANGLE  OR  DIRECTION  RADIUS  WEIGHT 

23  2  22 

LKTRKR:  0  CHAINS  OUTPUT  OF  MIN  LENGTH  20  -  0  LINKS  TOTAL 

GRADDL  ON  IMACE-4621-3  WINDOW-1100011248  7752  8000 

GRADDL : NOW  THIN  MAXMAG, TYPE-  141  0 

SELECT: THRESHOLD-  74 

SELECT : LENWRK—  62 

OSIGNIFIGANT  LOCAL  MAXIMA  WITH  THRESHOLD  -  20  WERE  FOUND  AT 

ANCLE  OR  DIRECTION  RADIUS  WEIGHT 
17  -2  27 

LFOCUS ; i 1159 ,  8007  11159,  7759  179,0,  -3,0,  WT-  22  20,32  20,  1 

LKTRKR:  0  CHAINS  OUTPUT  OF  NIN  LENGTH  20  -  0  LINKS  TOTAL 

***VIRTUAL***  11000  11248  77S2  8000*SCREEN*  1  32  1  32  ***** 

1234567890123456789012345678901234567890 


- - 


UbiUNit-  1LAN  I  LULAL  MAXIMA  W1IH  IMKtbHULD  -  WtKt  I-  UUNU  HI 

c 

ANGLE  00  DIRECTION  RADIUS  WEIGHT 
_  17  -2  27 

C  LFOCUSs 111S9,  0007  11159,  7759  179.0,  -3.0,  UT»  22  20,32  20,  1 

LKTRKRt  ft  CHAINS  OUTPUT  OR  MIN  LENGTH  2S  -  0  LINKS  TOTAL 

***VIRTUAL***  11000  11248  7752  8000*SCREEN*  1  32  1  32  ***** 

C  1234567890123456789012345678901234567890 


M 

32 

31 

r 

30 

u 

29 

28 

< 

27 

26 

X 

i  < 

25 

X  • 

.« 

24 

X  • 

v i 

23 

•  • 

4 

22 

*  #x 

»  t 

21 

*  *x 

* 

20 

*  ex 

19 

•  t 

1 

18 

• 

* 

17 

•  • 

16 

•  * 

f 

15 

•  * 

If 

14 

*X 

*  • 

13 

XX 

*  • 

€ 

12 

•  X 

•  • 

'' 

11 

88 

*  * 

♦ 

10 

8* 

X  • 

9 

•  •X 

•« 

<i 

8 

X 

•  * 

i 

7 

•« 

6 

•• 

n 

5 

**x 

•* 

4 

3 

2 

1 

GRADDL  ON  IMAGE«=4621-3  WINDOU»ll 0 001 1248  7952  8200 
GRADDL : NOW  THIN  MAXMAC , TYPE"  157  0 

SELECT :THRESHOLD=  80 
SELECT :LENURK=  59 

0SIGNIFIGANT  LOCAL  MAXIMA  WITH  THRESHOLD  -  20  MERE  FOUND  AT 


s 


)i 


)• 


i 


* 

41 


41 


ANGLE  OR  DIRECTION  RADIUS  WEIGHT 
17  -3  24 

LFOCUS: 11159,  8207  11159,  79S9  179.0, 

LKTRKRi  0  CHAINS  OUTPUT  OF  MIN  LENGTH  20 
***UIRTUAL***  11000  11248  7952  8200*SCREEN* 

1234567890123456789012345678901234567890 


32 

31 

30 

29 

28 

27 

26 

25 

24 

23 

22 

21 

20 

19 

18 

17 

16 

15 

14 

13 

12 

11 

10 

9 

8 

7 

6 

5 

4 


X 


XX 

XX 

X 

X  »x 

X  •» 

X  «x 

••  •  *• 

♦  *  X  *x 

*•  »x 

f  •*  «x 


*  **x 

•  MIX 

9 

XX 

•  t 

•  X 

•  • 

X  * 

X  X 
X 


-3.0,  UT=  14  20,32 

0  LINKS  TOTAL 
1  32  1  32  ***** 


20 ,  1 


20b 


3 


*  GRADDL  ON  IMAGE-4621-3  WINDOW- 1 1 00 0 1 1248  81S2  8400 

' C  GRADDL:NOW  THIN  MAXMAG, TYPE-  129  0 

♦  SELECT: THRESHOLD-  64  - 

SELECT sLENWRK-  69 


< 

0SIGNIFIGANT  LOCAL  MAXIMA  WITH  THRESHOLD 

* 

20 

WERE  FOUND 

AT 

t 

ANGLE  OR  DIRECTION 

RADIUS 

WEIGHT 

17 

-4 

45 

1 

2 

24 

LFOCUS: 11167,  8407 

11167, 

8159 

179 

•  c. 

-4.0, 

WT- 

24 

21,32 

21,  1 

t 

LFOCUS: 11143,  8159 

11143, 

8407 

-1 

.  0  > 

1.0, 

WT- 

24 

18,  1 

18,32 

»» 

LKTRKR : L INID , AX , AY 

.GLOBAL  AX, AY- 

i 

17  5 

11143 

8199 

23 

7  7  7  7  7  7 

7  7  7 

7  7 

7  7 

7 

7 

7  7  7 

7 

7 

t  777 

LKTRKR :  1  CHAINS  OUTPUT  OF  HIN  LENGTH  20  -  0  LINKS  TOTAL 

***UIRTUAL***  11000  11248  8152  B400*SCREEN*  1  32  1  32  ***** 


t  1234567890123456789012345678901234567890 


11 

32 

31 

( 

30 

14 

29 

28 

X 

X 

< 

27 

X 

X 

V 

26 

* 

9 

25 

« 

9  + 

< 

24 

« 

9  + 

10 

23 

9 

9X 

22 

* 

•  X 

( 

21 

• 

♦  X 

11 

20 

• 

•  X 

( 

19 

» 

•  X 

18 

* 

♦  X 

1* 

17 

* 

OX 

16 

• 

ox 

( 

15 

• 

BX 

19 

14 

• 

ex 

13 

• 

ex 

12 

• 

ex 

41 

11 

• 

ex 

< 

10 

• 

ex 

9 

• 

ex 

if 

8 

9 

ox 

7 

9 

ox 

6 

9 

ox 

<1 

5 

9 

ex 

4 

<-  3 

2 
1 

*■  GRADDL  ON  IMAGE-4621-3  WINDOW-10800 1 1 048  8152  8400 

GRADDL i NOW  THIN  HAXHAG,TYPE-  114  0 

SELECT i THRESHOLD-  10 
'  SELECT iLENWRK-  80 

•LBCANi  NO  LOCAL  MAXIMA  FOUND  WITH  THRESH  -  28 

LKTRKR:  1  CHAINS  OUTPUT  OF  MIN  LENGTH  20  -  0  LINKS  TOTAL 

(  GRADDL  ON  IMAGE-4621-3  WINDOW- 1 060 0 1 0848  8152  8400 

GRADDL : NOW  THIN  MAXMAG, TYPE-  154  O 
SELECT .THRESHOLD-  73 
1  SELECT tLENWRK-  35 

OSIGNIFICANT  LOCAL  MAXIMA  WITH  THRESHOLD  -  20  WERE  FOUND  AT 

f  ANGLE  OR  DIRECTION  RADIUS  WEIGHT 

11  4  24 
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4 


s 


p 


61 


CRADDL;NOW  THIN  MAXMAG.TYPE-  ISA 
SELECT : THRESHOLD-  73 
SELECT :LENWRK«  35 
OSIGNIFIGANT  LOCAL  MAXIMA  WITH  THRESHOLD  -  20  WERE  FOUND  AT 


1 

4 

« 


( 

( 

i 

i 

( 


M 


'0 


n 


n 


4; 


247.0, 

2  S 

12  11 


ANGLE  OR  DIRECTION  RADIUS  WEIGHT 
11  4  24 

LFOCUS : 10607,  8303  1085S,  8199 

LKTRKR:LINID, AX, AY, GLOBAL  AX , AY- 
121212121122 
1  2  1 

LKTRKR :  2  CHAINS  OUTPUT  OF  MIN  LENGTH  20 

**«UIRTUAL*«*  10600  10848  8152  84004SCREEN* 

1234567890123456789012345678901234567890 
32 
31 
30 
29 
28 
27 
26 
2S 
24 
23 

21 
20 
19 
18 
17 
16 
IS 
14 
13 
12 
11 
10 
9 
8 
7 

6  X 

5  X 

4 

3 

2 

1 

PATTERN  COMPLETED  9  WINDOWS  SAMPLED 
WANT  TO  CONTINUE  SCAN  PATTERN?  ( YES= 1 /NO=0 ) - 
0  , 

IERR=  0  IN  CLOSING  EEFILE 


,  0,  WT-  16 
17  10647 

2  2  1 


1,19 

8295 


32,  6 
23 


1  32 


LINKS  TOTAL 

1  32  *»**» 


•  « 


8* 


«* 


8* 


*•« 


*• 


98 


*XX  + 


XX  + 


x+ 
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C.  3 _ Loader 

Command 

RE,%EDGEY 

RE,%RPSCH 

RE , %ARRAY 

RE  ,%RPSFX 

RE,7,RPSGL 

RE,%RPSHG 

RE,%RPSFO 

RE ,7RPSLK 

RE,%RPSLS 

RE,%RPSPR 

RE,%RPSPA 

RE,7oRPSSE 

RE,%RPSWR 

RE,%RPSVT 

RE,%RPSHS 

RE,%RPSKL 

RE,%RPSVG 

RE, % STAGE 

RE.7.RPSFR 

RE,%RPSMK 

RE , %INSPT 

RE , % VEREE 

RE , 7  I NS  EC 

RE, %RPSAA 


commands  for  creating  EDGEY 

Comment 
main  routine 

block  data  routine  for  chain  code  common 
window  sampling  routine 
routine  initializes  accumulator  array 
computes  32-direction  gradient 
computes  Hough  transform 
focuses  Hough  transform 
extracts  continuous  curves 
scans  accumulators  for  peaks 
prints  grey  scales  if  required 
prints  accumulators  if  required 
selects  strong  gradient  points 
links  edge  points  to  best  neighbors 
converts  vector  to  gradient  direction 
histograms  gradient 
computes  gradient  at  single  point 
converts  polar  vector  to  rectangular 
positions  stage 
sets  up  masks  for  given  edge 
maintains  and  applies  gradient  masks 
inspects  both  sides  of  edge  for  other  edges 
verifies  previously  detected  edge 
computes  intersection  between  2  lines 
worker  subroutine  for  RPSLS 
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M  (n  ni  ill  1 1  i>u  in  in 
•Dll  f  EDUMJ.  ;  JH.ll  DM  K  :  :  1  |l 
RUN  ,1  D1IH' 

source  Fin? 

/f  Fnr.F  k  :  tn 

D«tUx  ORG* 10000  500  0fUR(Nl  =  .  05WGT--  20  IS 

/♦ 


LK 

Rill  , 

2733 

16  . 

7  .  ^  .  6,  6 ,  7 . 

6  ,  6  ,  6  ,  6  , 

7,  6,  6 ,  6,  6,  6  7 

■90 

FO 

6943  , 

4885  . 

7023  . 

4637  . 

*5 

FO 

9637  , 

5209  . 

9885  , 

5281  , 

LK 

9677  . 

5225  , 

23  . 

1  .  8  .  1  .  1  .  8  . 

1  .  1  ,  B  .  1  . 

1  ,  1  ,  0  .  1  ,  1  ,  1  .  0  ,  1  .  1 

1.1? 

/-900 

EOF 

/I 


PREUX 

ORG- 1 ( OOP 

5000PERCNT* 

.  OSWGT- 

20 

IS 

/ L  50 
PREUX 

ORG-IOOOO 

50  0  OPERCNT  * 

,  OSWGT- 

20 

15 

FO  6943, 
FO  2201 , 
FO  3885, 

4885, 
Si  35. 
5103, 

7023,  4637, 
2281,  4887, 
3637,  49 59, 

l_K  1  0677  . 

4927, 

17,20,20 , 

8  * 

B  .20  , 

8,20,20  , 

0.  8. 

8.20, 

8.20, 

8  , 

8.10, 

LK 10677, 
FO  9637. 
EOF 

5201  . 
5209  , 

21  2  ° 
Q885 .  5281 . 

*■ ' 

1  ,  1  , 

°  2  2 

'>  2 

2  ,  1  , 

o  2 

2 ,  1,23,  6  20.  6 

/ER 

ENT  OF  EDIT 
: RUN, EDGE T. 6 .6 
GIVE  IMAGE  NAMF<3A?>- 
DRirUX 

GIVE  FRACTION  OF  POINTS  TO  SELECT ( RE At /FR EE > - 
0  .  OS, 

GIVE  10  I/O  FLAGS  (1NT/FREE>- 
0,0. 0,0,  0.0  .0.0.  0.0, 

GIVE  GLOBAL  ORIGIN  AND  STAGE  BOUNDS  (6  INT /FREE )  - 
100  00 .SO  0  0 .8 SO ,2700 . 11200 .920  0  , 

*  PTS  ON  LINE,  ON  CHAIN,  6  TRACKING  SIZE  (3  I  NT /FREE ) * 

IS. IS, 150 , 

IERR=  4  IN  OPENING  FILE  EEFILE 

IERR=  4  IN  OPENING  FILE  EEDGEX 

1ERR.LEN*  0  23 

EEFILE  REC* 

DREUX  ORG*l  000  0  S000PERCNT*  .  05UGT*  20  IS 
IEPR.LEN*  0  13 

EEFILE  REC* 

FO  6943,  4885,  7023,  4637, 

GRADDL:  ON  DREUX  WINDOW-  6063  7111  4641  4889  MAXMAG , T YPE  =  6?  0 

SELECT: THRSHL .LENWRK-  29  31 

OSIGNIFIGANT  LOCAl  MAXIMA  WITH  THRESHOLD  =  15  WFRE  FOUND  AT 

ANGLE  OR  DIRECTION  RADIUS  WEIGHT 

15  2  31 

I.FOCUS:  6934,  4R96  7030,  4648  200  0,  2-0.  HT=  20  9.32  21.  1 

GRADDL:  ON  DREUX  WINDOW-  6934  7182  4693  4941  MAXMAG, TYPE-  21  0  x 

SELECT  :  THRSHL ,L  ENURK -  0  il 

01  SCAN:  NO  LOCAL  MAXIMA  FOUND  WITH  THRESH  ■  15 

GRADDL :  ON  DREUX  WINDOW-  6750  6998  4693  4941  MAXMAG,  TYPF-  31  0 

SELECT :  THRSHL .LENWRK-  8  17 

OLSCAN:  NO  LOCAL  MAXIMA  FOUND  WITH  THRESH  -  15 

GRADDL:  ON  DREUX  WINDOW*  6980  7228  4550  4798  MAXMAG. TYPE*  22  0 

5ELECT : THRSHL, LENWRK*  8  9 

OLSCAN:  NO  LOCAL  MAXIMA  FOUND  WITH  THRESH  a  15 
GRADDI  :  ON  DREUX  WINDOW*  6796  7044  4550  4798  MAXMAG  ,  TYPE*  30  0 

SELECT:  THRSHL. LENWRK*  9  34 

OLSCAN:  NO  LOCAL  MAXIMA  FOUND  WITH  THRESH  *  15 

GRADDI  :  ON  DREUX  WINDOW*  6943  7191  4393  4641  MAXMAG. TYPE-  45  0 

SELECT : THRSHL .LENWRK*  2S  32 

nSIGNIFICANT  LOCAL  MAXIMA  WITH  THRESHOLD  *  15  WFRE  FOUND  AT 

ANGLE  OR  DIRECTION  RADIUS  WEIGHT 

15  -1  32 

L FOCUS :  7038,  4648  7142,  4400  202.0,  -1.0.  UT-  22  12,32  2S ,  1 

FORWARD  VERFE : TAX  ,  TAY , TBX  ,TBY .KFLAG*  7023  4637  7103  4389  1 

GRADDL.  ON  DREUX  WINDOW*  7014  7262  444S  4693  MAXMAG , T YP F  3  17  0 

5ELECT : THRSHL .LENWRK*  8  12 

OLSCAN:  NO  LOCAL  MAXIMA  FOUND  UIT  I  THRESH  -  15 

GRADDL:  ON  PREUX  WINDOW*  6830  707(3  4445  4693  MAXMAG  .  T  YPE  *  2  6  0 

SELECT  •  THRSHL .LENWRK*  0  22 

OLSCAN'  NO  LOCAL  MAXIMA  FOUND  WITH  THRESH  *  IS 
GRADDL  ON  DREUX  WINDOW*  7060  7300  4302  4550  MAXMAG  T  YPF  =  23  0 

SELECT • THRSHL .LFNURK*  A  13 
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is 

4*. so  maxmag  .  i  m  - 


III  fif  AN  :  Nil  I  1 1C  A I  MAXIMA  F  (HIND  WITH  THHt  HM  *  15 

UN  At>hl  .  > IN  PtttlU  UINDHW-  AH7A  M?4  4  111.*  4*.S0  MAXMAG  .  I  m  - 

nri  M  t  .  iimnni  i  i  nuwh-  m  i  .* 

Ill  "it  AN  I  Nil  HUAI  MAXIMA  F  01  INI)  WITH  THRESH  -  IS 
GRADDL  :  ON  PRFUX  WINDOW-  70?  3  7271  4145  4  393  MAXMAG  .  T  TPF  - 

SELECT : 1 HRSHL . LENWRK  *  29  32 

OSIGNtFlGANT  LOCAL  MAXIMA  WITH  THRESHOLD  *  IS  WERE  FOUND  AT 


ANGLE-  OR  DIRECTION  RADIUS  WEIGHT 
IS  -2  32 

LFOCUS:  7134,  4400  7222,  4152 

FORWARD  UERFE : TAX ,TAY ,TBX ,TBY , KFLAC- 
GRADDL  :  ON  DREUX  WINDOW-  7094  7342 


200.0,  -2.0,  WT«  21  1 

7103  4389  7103  4141 

4197  444S  MAXMAG , TYPE* 


4302  MAXMAG . TYPE- 


4302  MAXMAG, TYPE= 


4145  MAXMAG, TYPE* 


15  WERE  FOUND  AT 


SELECT-.  THRSHL, LENWRK-  B  IS 

OLSCAN:  NO  LOCAL  MAXIMA  FOUND  WITH  THRESH  *  15 

GRADDL:  ON  DREUX  WINDOW-  6910  7158  4197  4445  MAXMAG, TYPE*  36 

SELECT: THRSHL, LENWRK-  0  30 

OLSCAN:  NO  LOCAL  MAXIMA  FOUND  WITH  THRESH  *  15 

GRADDL:  ON  DREUX  WINDOW-  7140  7388  4054  4302  MAXMAG. TYPE*  30 

SELECT: THRSHL .LENWRK-  8  25 

OLSCAN:  NQ  LOCAL  MAXIMA  FOUND  WITH  THRESH  «  15 

GRADDL:  ON  DREUX  WINDOW-  6956  7204  4054  4302  MAXMAG, TYPE*  ?R 

SELECT : THRSHL , LENWRK*  0  29 

OLSCAN:  NO  LOCAL  MAXIMA  FOUND  WITH  THRESH  *  15 

GPADOI  :  ON  DREUX  WINDOW*  7103  7351  3897  4145  MAXMAG, TYPE*  04 

SELECT : THRSHL .LENWRK-  IS  42 

OSIGNIFIGANT  LOCAL  MAXIMA  WITH  THRESHOLD  *  15  WERE  FOUND  AT 

ANGLE  OR  DIRECTION  RADIUS  WEIGHT 

IS  -4  37 

LFOCUS {  7222,  4152  7334,  3904  204.0,  -4.0,  WT»  10  15,3? 

FORWARD  UEREE : TAX , TAY , TBX , TBY , KFLAG-  7183  4141  7263  3893  1 

GRADDL;  ON  DREUX  WINDOW*  7174  7422  3949  4197  MAXMAG .TYPE*  70 

SELECT: THRSHL, LENWRK-  IS  59 

OSIGNIFIGANT  LOCAL  MAXIMA  WITH  THRESHOLD  -  15  WERE  FOUND  AT 

ANGLE  OR  DIRECTION  RADIUS  WEIGHT 

20  -9  27 

LFOCUS :  7429,  4124  7301,  3956  143. 0.  -9.0.  WT-  14  32.22 

***UXRTUAL***  7174  7422  3949  4197*SCREEN*  1  32  13?  ***** 

1234567890123456789012345670901234567890 
32 


X  ♦/ 

X  +  / 

*  *X^- 

*  X 
$♦ 

♦  sx/ 

t  X/- 
X/ 

X  4- 

x+ 

X/ 

/  ♦//  /  8 


INSEC  RETURNS 
7183  4141 


AX , AY , BX ,BY,CX,CY,DX ,DY,IX, IY.ITYPE- 
7229  3998  7429  4140  7293  3956  7258  3908 


Appendix  D 


_ Registration  Software 


The  registration  software  consists  of  one  load  module  containing 
one  main  program  and  several  subroutines.  The  program  structure  is 
illustrated  below: 


The  registration  program  takes  a  set  of  image  edges  from  the 
file  IMAGE  and  a  set  of  map  edges  from  the  file  MAP  and  computes  the 
possible  transformations  that  will  transform  the  image  onto  the  map. 
The  technique  used  is  to  cluster  the  possible  transformations  in 
3-dimensional  space  and  select  those  with  the  strongest  support. 

The  data  structures  used  reside  in  the  four  common  blocks, 
presented  in  D.l  ,  and  the  match  weight  matrix,  MATCHS(200,30)  is 
in  the  common  block  MATRX  which  resides  in  the  RTE-IV,  extended  memory 
facility. 
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Routine 

Name 


REGDR 


RDCNTR 


CNVTRP 


EDGMAP 


SCABKT 


TP 


Arguments 


Function 


Main  Program  Reads  and  analyzes  commands  and  calls 

appropriate  subroutine.  Sets  up  initial 
values  for  variables  contained  in  com¬ 
mon  blocks  found  in  the  procs:  REGIST, 
BUCKET,  and  PRTFLG. 


(NUMLEV,SCALEX,  Reads  in  the  edge  information  for  the 

SCALEY,SCALET,LOXBND,  image  and  then  the  map  from  DISK  FILES. 

LOYBND,LOTBND)  Image  edge  information  must  reside 

in  the  file  IMAGE: :18.  Map  edge  infor¬ 
mation  must  reside  in  MAP::18.  Computes 
centers  of  image  and  map  windows  and  places 
them  in  common  block.  Using  NUMLEV, 
computes  the  three  scales  at  each  level 
and  stores  them  in  arrays  SCALEX.SCALEY 
and  SCALET.  Computes  lower  bounds  at 
first  level  and  stores  them  in  LOXBND(l) , 
LOYBND(l)  and  LOTBND (1 ) .  Image  edge 
information  stored  in  arrays  X1,X2,Y1,Y2 
of  common  block  REGIS.  Map  edge  infor¬ 
mation  into  arrays  U1,U2,V1,V2  of  same 
common  block. 


(XI, Y1,X2,Y2, RADIUS,  Takes  a  directed  straight  line  segment 
THETA)  beginning  at  (XI, Yl)  and  ending  at 

(X2,Y2)  and  converts  it  to  polar  coor¬ 
dinates  (RADIUS, THETA) . 


(X1,Y1,X2,Y2,U1,V1, 

U2 , V2 , STOL .THETAR, 
STAILX , STAILY , SHEADX , 
SHEADY) 


Given  an  image  edge  defined  by  (XI, Yl) 
and  (X2,Y2),  a  map  edge  defined  by 
(U1,V1)  and  (U2,V2),  an  angle  of  rota¬ 
tion  of  image  to  map  edge  (THETAR)  and 
a  tolerance  (STOL) ;  a  line  segment  in 
a-space  which  represents  the  constraints 
of  the  x-shift  and  the  y-shift  in  trans¬ 
formations  on  these  two  edges  is 
calculated  and  is  represented  by  (STAILX, 
STAILY)  and  (SHEADX, SHEADY) . 


(TAILX,TAILY,HEADX, 
HEADY , THETA I , THEIND , 
SCALX , LOX , SCALY , LOY , 
MATNUM , OUTUNT ) 


Creates  the  clustering  matrices  used  to 
find  the  most  relevant  transformations. 
Since  two  sets  of  clustering  matrices 
are  used  by  the  approach  (regular  and 
offset  matrices)  the  routine  is  called 
twice  for  each  line  segment  it  processes. 
The  line  segment  is  defined  by  (TAILX, 
TAILY),(HEADX, HEADY)  and  the  angle  of 
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Routine 

Name 


Arguments 


Function 


the  line  is  in  THETAI .  MATNUM  indicates 
whether  incrementation  is  to  be  done  in 
the  regular  matrix  (MATNUM=1)  or  the 
offset  matrix  (MATNUM=2) .  The  scale  of 
the  matrix  is  defined  by  SCALX.LOX, SCALY 
and  LOY.  The  line  is  chased  from  tail 
to  head  and  each  bucket  of  the  matrix  that 
it  passes  through  is  incremented.  Infor¬ 
mation  about  each  line  is  written  to  an 
intermediate  output  file.  Clustering 
matrices  BKTCNT  &  BKTOFF  are  in  the 
common  block  in  proc  BUCKET. 


STRAIT 

(AX, AY, BX, BY, 

NNEIGB , STORAG , N , NLINKS , 
IFLAG) 

Given  a  line  defined  by  (AX, AY)  and 
(BX,BY)  and  whether  it  is  to  use  8- 
directional  or  4-direction  Freeman  codes, 
STRAIT  computes  the  links  along  that 
Straight  line.  The  links  are  stored  in 
the  array  STORAG (N) .  If  the  number  of 
links,  NLINK,  is  greater  than  N,  IFLAG 
is  set  to  indicate  an  error. 

BEST 

(FX,FY,TX,TY, NNEIGB 

L1,L2 ,N1,N2) 

Given  a  line  from  (FX.FYJ  to  (TX.TY); 
computes  the  straightest  path,  returning 
it  of  LI  links  in  N1  and  it  of  L2  links 
in  N2. 

CHOSPK 

(PEAKS,  MAXPKS , NUMPKS , 
PRTFLG, PWIDTH, THRESH) 

Scans  the  two  clustering  matrices, 

BKTCNT  and  BKTOFF,  to  find  the  MAXPKS 
highest  values.  The  indices  of  the  high 
valued  buckets  are  stored  in  the  array 
PEAKS  with  the  number  of  peaks  actually 
chosen  being  in  NUMPKS.  THRESH  defines 
the  minimum  value  to  be  considered  as  a 
possible  peak.  PRTFLG  and  PWIDTH  provide 
information  to  the  print  routines. 

ALMTCH 

(THETA, XSHIFT.YSHIFT, 
ANGTOL , DTOL , PWIDTH , 
LENCHK , MTCHWT , 

NMCHRW , NMCHCL , COLMCH) 

Given  a  transformation  defined  by 
(THETA, XSHIFT, YSHIFT) ,  an  angle  tolerance 
of  ANGTOL  and  a  distance  tolerance  of 
DTOL;  an  evaluation  of  how  good  the  trans 

formation  is  made.  This  is  done  by 
transforming  any  image  edge  that  possess 
approximately  the  correct  gradient  onto 
a  map  edge.  The  function  EMATCH  is 


214 


Routine 

Name 


Arguments 


Funct ion 


EMATCH 


Worker 

STACK 

PRTBKT 

PRTMAT 

RDWRTF 

SGN 


called  to  actually  evaluate  how  good 
each  match  is.  Taking  the  results  of 
EMATCH,  ALMTCH  creates  the  match  weight 
matrix  matchs  (in  common  block  MATRX) , 
the  average  match  weight,  MTCHWT,  the 
number  of  matching  rows,  NMCHRW,  the 
number  of  matching  columns,  NMCHCL,  and  in 
which  columns  a  match  has  been  found. 

If  LENCHK=1,  only  edges  of  approximately 
the  same  length  are  compared.  PWIDTH 
is  used  by  available  .output  routine. 


(SEG1.SEG2, GRAD, THETA,  Given  the  transformation  (THETA, XSHIFT, 
XSHI FT , YSHIFT , DTOL)  YSHIFT) ,  the  edge  in  the  image  (stored 

in  array  SEG1) ,  the  edge  in  the  map 
(stored  in  array  SEG2) ,  the  gradient 
direction  of  the  map  edge,  GRAD,  the 
gradient  direction  of  the  image  edge, 
THETA,  and  the  tolerance,  DU)L; 
computes  strength  of  match  and  returns 
a  value  between  0  and  1, 


routines  not  shown  in  program  structure. 


(CODE, PEAKS, I) 


(PWIDTH, CNTSUM) 


(RSIZE,CSIZE .NIMAGE, 
NMAP,  PWIDTH) 

(FILENO, OPCODE, 
BUFFER, IL.IERR, 

TYPE) 


(U 


Maintains  a  push  down  stack  thac  keeps 
track  of  peaks  to  be  evaluated.  Each 
entry  contains  5  fields;  THETA  INDEX, 
X-INDEX,  Y-INDEX,  WEIGHT  and  which  matrix 
peak  appeared  in. 

Prints  either  the  original  or  offset 
clustering  matrix  depending  on  the 
value  of  CNTSUM.  Matrices  in  common 
block  BUCKET 

Prints  the  match  weight  matrix  created 
in  ALMTCH. 

Controls  the  input  and  output  to  the 
intermediate  disk  files.  Possible 
operations  are  OPEN,  READ, WRITE, REWIND, 
and  CLOSE.  Can  have  up  to  2  files  open 
at  once. 

Returns  1  if  1^0,  -1  if  1^0  and  0  if 

1=0. 
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D.i  Procs  containing  common  blocks  used  in  registration 


C 

C  RE  GIST  PROC******H.P ,  REGISTRATION  CODE  9FEB79***#*  i 

C 

INTEGER  XI (200)  ,X2<  200)  ,Y  1(200)  ,Y2<  200)  ,U.U30)  , 

*  U2( 30 ) , Vi ( 30 ) ,V2<30) , NIMAGE ,NMAP  , IXCENT , IYCENT , MXCENT , 

K  MYCENT ,  RSI ZE , CSIZE 

REAL  THET(2,200> ,RAD<2,200) 

C 

c 

C  X 1  ,  X2 , Yi , Y2  -  ARRAYS  FOR  ENDPOINTS  OF  IMAGE  LINES 

C  111  ,  U2  ,  Vi  ,  V2  -  ARRAYS  FOR  ENDPOINTS  OF  MAP  LINES 

C  THET  -  ANGLES  FOR  IMAGE  AND  MAP 

C  RAD  -  RAD  FOR  IMAGE  AND  MAP 

C  NIMAGE  -  #  OF  LINES  IN  IMAGE  <*  OF  ROUS  IN  MATRIX) 

C  NMAP  -  *  OF  LINES  IN  MAP  <#  OF  COLUMNS  IN  MATRIX) 

C  IXCENT , IYCENT  -  X  AND  Y  OF  IMAGE  CENTER 

C  MXCENT , MYCENT  -  X  AND  Y  OF  MAP  CENTER 

C  RSIZE  -  *  OF  ROWS  IN  MATCHS 

C  CSIZE  -  *  OF  COLUMNS  IN  MATCHS 

C 

C  NAMED  COMMON  REGIS  IS  USED  TO  HOLD  INFO 

C 

COMMON  /REGIS/  Xi , X2 , Y1 , Y2 , Ui  , UP , V 1 ,02  , THF T  , RAD  , 

*  NIMAGE }NMAP , IXCENT .IYCENT  , MXCENT, MYCENT , RSIZE, CSIZE 

C 

C***  END  OF  MACRO  REGIST,  H .  P  .  VERSION  9FEB79**#** 

C 

C 

C****#PROC  DUCKET  H.P.  VERSION  9FEB79#*##**# 

C 

C  DEFINFS  CLUSTER  MATRICES,  CLUSTERING  IN  3-D  ( THETA ,X,Y> 

C  BKTCNT  -  CONTAINS  COUNT  OF  #  OF  HITS  IN  EACH  BUCKET 

C  BKTOFF  -  CONTAINS  COUNT  OF  #  OF  HITS  IN  OFFSET  MATRIX 

C 

INTEGER  BKTCNT ( 10  ,10  ,  i 0 ) , BKTOFF ( 10  ,10,10) 

INTEGER  NZEROT (10) , NUMTHE , NUMX , NUMY , NZEROO (10) 

COMMON  /OLSTR /BKTCNT , BKTOFF , NZEROT , NZEROO , NUMTHE , NUMX , NUMY 
C 

C***  END  OF  PROC  BUCKET  **# 

C 
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u 

C*#***PRTFLG  MACRO  H  .  P  .  VERSION  9FEB79***** 

C 

INTEGER  PRTINP ,PRTMWM .PRTBUK , PRTSBK 
C 

C  PRTINP  -  PRINT  INPUT  FLAG  (RECTANGULAR  AND  POLAR) 

C  PRTMWM  -  PRINT  MATCH  WEIGHT  MATRIX  FLAG 

C  PRTBIJK  -  PRINT  3-D  CLUSTERING  MATRIX  FLAG  (BUCKETS) 

C  PRTSBK  -  PRINT  SMOOTHED  CLUSTERING  MATRIX 

C 

C  NAMED  COMMON  PRTFL  IS  USED  TO  HOLD  FLAGS 

e 

COMMON  /PRTFI  /  PRTINP  , PRTMWM , PRTBUK , PRTSBK 

C 

C***  END  OF  PROC  PRTFLG 

C 

C 

C#****PROC  CHSDF8  H.P.  VERSION  9FEB79**##**** 

C 

INTEGER  DX(8) ,DY(8) 

C 

C  DX.DY  -  CHANGES  IN  X  AND  Y  FOR  CHAIN  CODES 

C  THE  NAMED  COMMON  DF.LTS  HOLDS  THESE  VALUES  AND  IS  INITIALIZED 
C  BLOCK  DATA  PROGRAM 

C 

COMMON  /DELTS/  DX,DY 

C 

C**#  END  OF  PROC  CHSDF8 

C 


IN  THE 
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D. 2  Possible  commands 


The  command  needed  to  get  things  started  is  the  INPUT  command. 
The  format  is  as  follows  (starting  in  column  1) : 


INPUT  ^NGTOL^,  <£tOI>  ,<DTOL>,  <&UMLeO>  <£rTBUk) , <PRTOF^> ,  <£rTMWm), 

^PWIDThX  <J.ENCHK> ,  ^THRESH^ 


Where  the  input  fields  have  the  following  significance: 

ANGTOL  -  angular  tolerance  (in  degrees)  between  model  edge  and  a  rotated 
image  edge.  Used  in  the  routine  ALMTCH. 

STOL  -  segment  tolerance  for  generating  the  line  segments  in  EDGMAP. 

DTOL  -  distance  tolerance  used  in  computing  strength  of  match  between 

a  model  edge  and  a  transformed  image  edge.  Calculation  using 
DTOL  is  in  routine  EMATCH. 

NUMLEV  -  number  of  clustering  levels  desired.  Maximum  currently  possible 
is  5.  Used  as  a  controlling  parameter  in  REGDR. 

PRTBUK  -  flag  for  printing  original  clustering  matrix.  If  value  is  1, 
matrix  is  printed  at  each  level. 

PRTOFF  -  flag  for  printing  offset  clustering  matrix.  If  value  is  1, 
matrix  is  printed  at  each  level. 

PRTMWM  -  flag  for  printing  match  weight  matrix. 

PWIDTH  -  number  of  columns  available  on  output  line.  Can  be  either  72 

or  132. 

LENCHK  -  if  set  to  1,  only  edges  whose  lengths  are  approximately  the 
same  will  be  compared. 

THRESH  -  minimum  number  of  line  segments  that  must  pass  through  a  bucket, 
at  the  highest  clustering  level,  in  order  for  that  bucket  to  be 
considered  as  a  peak. 

All  above  inputs  are  integer. 
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The  command  to  initiate  the  3-dimensional  clustering  that  does 


the  bulk  of  the  registration  work,  starts  in  column  one  and  is  CLUSTR. 

A  command  that  can  be  presented  independently  of  the  CLUSTR 
command,  is  the  EMATCH  command.  This  command  allows  the  user  to  present 
a  possible  transformation  and  have  the  software  evaluate  the  strength  of 
the  transformation  given  the  edge  information  input  as  a  result  of  the 
INPUT  command  processing.  This  command  can  be  given  before,  after  or  in 
place  of  the  CLUSTR  command.  The  format  is: 

EMATCH  <THETAI> ,  <XS> ,  <YS> ,  <DTOI^ 

where 

THETAI  -  rotational  angle,  in  degrees,  of  the  transformation 
to  be  evaluated,  (integer) 

XS  -  X-shift  of  transformation  to  be  evaluated  (real) 

YS  -  Y-shift  of  transformation  to  be  evaluated  (real) 

DTOL  -  distance  tolerance  to  be  used  in  computing  strength 
of  matchs  for  this  transformation,  (integer) 

The  command  to  terminate  processing  is  simply  FINISH. 
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Appendix  E  Verification  Software 

The  software  to  do  the  verification  of  lineal  features  has  been 
set  up  as  an  independent  load  module.  The  main  program  of  this  system 
is  the  program  VERIFI.  The  software  has  the  ability  to  verify  both 
straight  and  curved  lines.  For  curved  lines,  the  input  can  either  be 
presented  one  point  at  a  time  or  the  information  can  be  stored  in  a  data 
file  which  will  be  processed  by  the  software. 

The  program  structure  is: 


The  verification  program,  given  an  inverse  transformation, 
will  check  the  image  for  the  presence  of  known  map  features. 
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Routine 

Name 


VERIFI 


ARRAY 


STRAIT 


BEST 


MLTDIR 


Arguments 


Function 


main  program 


(DELTAX, DELTAY , 
SYVXLO , SYVYLO) 


(AX,AY,BX,BY,NNEIGB, 

STORAG,N,NLINKS, 

I FLAG) 


(FX, FY , TX, TY , NNEIGB , 
L1,L2,N1,N2) 


(X,Y,RES,MAG,  DIR, 
IERR) 


Receives  user  input  which  defines  the 
Inverse  transformation  and  the  features 
to  be  verified.  The  user  is  prompted 
for  the  needed  information.  (An  example 
execution  is  presented  below)  For  each 
point  on  the  hypothesized  feature,  a 
profile  is  created  and  the  gradients 
along  the  profile  are  checked  to  see 
if  the  feature  truly  exists  at  this 
point.  Execution  can  continue  for 
several  edges  for  the  given  transformation. 

Given  the  change  to  be  made  in  the 
X-direction  (DELTAX)  and  in  the  Y- 
direction  (DELTAY),  the  stage  is  moved 
the  desired  amount  and  the  coordinates 
of  the  point  where  the  stage  is 
focused  is  returned  in  SYVXLO  and 
SYVYLO. 

Given  the  tail  (AX, AY) ,  the  head 
(BX,BY)  and  the  number  of  directions  to 
use  (4  or  8) ,  the  links  of  a  straight 
line  between  the  tail  and  head  are 
placed  into  the  array  STORAG.  If  the 
number  of  links  (NLINKS)  is  greater 
than  the  value  of  N,  IFLAG  is  set  to 
indicate  an  error.  In  this  case, 

STRAIT  is  used  to  create  the  profiles. 

Calculates  the  straightest  way  to  get 
from  (FX,FY)  to  (TX,TY)  depending  on 
number  of  directions  available  for  use 
(4  or  8) .  . 

See  Appendix  B  for  description  of 
function. 
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The  following  are  examples  of  3  executions  of  the  verification 
software.  Computer  prompts  are  identified  by  a  CP  in  the  left  margin  and 
user  responses  by  a  UR  in  the  left  margin. 


Example  1:  Curved  line  with  point  information  stored  in  disk  file. 


UR  RUN  VERII.6 ,6 

CR  POSITION  STAGE  TO  INITIAL  POINT  (BY  HAND)  AND  INPUT  COORDINATES 

CR  FORMAT  IS  XPT.YPT  (INT/FREE) 

UR  10000,5000, 

CR  GIVE  VALUE  FOR  DT0L  (INT/FREE) 

UR  100, 

CR  GIVE  GRADIENT  THRESHOLD  VALUE  (INT/FREE) 

UR  4, 

CR  GIVE  DEBUG  FLAG  (VALUE  BETW  0  &  4) 

UR  3 

CR  GIVE  REVTHE , REVXS , REVY S , IXC ENT , I Y CENT , MXCENT , MY CENT 

UR  342,-2433. ,1701. ,8000,8000,11000,11000, 

CR  DTOL, REVTHE, REVXS, REVYX  *  100  342  -2433.  1701. 

CR  IXCENT.IYCENT, MXCENT, MYCENT  =  8000  8000  11000  11000 

CR  GIVE  EDGE  it 

UR  1 

CR  VERIFYING  EDGE  it  1 

CR  IS  THIS  A  STRAIGHT  LINE?  INPUT  1  FOR  YES ,  0  FOR  NO 

UR  0 

CR  IS  INPUT  IN  A  FILE?  (1  FOR  YES,  0  FOR  NO) 

UR  1 

CR  GIVE  FILE  NAME  OF  INPUT  FILE  -  (3A2) 

UR  CTDRU0 
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When  processing  of  edge  is  completed  and  statistics  have  been 
printed,  user  will  again  be  prompted  for  edge  nVimber.  If  there  are  no 
more  edges  to  be  processed  input  a  -1.  User  will  then  be  prompted  for 
another  reverse  transformation,  if  none  to  be  given,  again  input  a  -1. 


Example  2:  Curved  line  with^point  information  given  by  user. 


First  17  lines  are  the  same  as  example  1. 


CR  IS  THIS  A  STRAIGHT  LINE?  INPUT  1  for  YES,  0  FOR  NO 

UR  0 

CR  IS  INPUT  IN  A  FILE?  (1  FOR  YES,  0  FOR  NO) 

UR  0 

CR  GIVE  X,Y  AND  GRAD  ANGLE  VALUES  (-1,-1,-1,  FOR  ENDING) 

UR  11032,10046,154, 

.  give  one  point  at  a  time 

CR.  GIVE  X, Y  AND  GRAD  ANGLE  VALUES  (-1,-1,-!,  FOR  ENDING) 

UR  -1,-1,-1 

Stopping  same  as  in  example  1. 
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Example  3:  Straight  line  with  equally  spaced  sample  points. 


First  17  lines  are  the  same  as  example  1. 


CR 

IS  THIS  A  STRAIGHT  LINE?  INPUT  1  FOR 

YES,  0  FOR  NO 

UR 

1 

CR 

GIVE  XTAIL.YTAIL  (INT/FREE) 

UR 

4996,7282, 

CR 

GIVE  DELTAX,DELTAY , GRAD, AN G  AND  NUMPTS 

( REAL , REAL , INT , INT ) 

UR 

55.65,113.32,334,41, 

Stopping  is  accomplished  in  the  same  manner  as  in  example  1. 
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FOR  J  AND  EACH  LINE  SEGMENT  PAIRED  WITH  J  (BUT  NOT 

WITH  I  OR  ITS  PAIRS) 

CHECK  FOR  THE  DIFFERENT  CASES  OF  INTERSECTION. 
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